{"instance_id": "locomo_conv-50_q086", "cell": "no-context", "question": "What sports activity is Calvin planning to try after the tour with Frank Ocean?", "opus_verdict": "incorrect", "opus_failure_mode": "F3", "gpt_verdict": "incorrect", "gpt_failure_mode": "F3", "model_answer": "Surfing", "ground_truth": "Skiing", "context": "Session 9 (3:15 pm on 21 June, 2023):\nCalvin: Nope, not yet! I haven't been there before, but it's on my to-do list for after my tour with Frank Ocean ends. Can't wait to see them in person! Look at that snowy peak. [image: a photo of a person on skis on a snowy mountain]"}
{"instance_id": "locomo_conv-26_q059", "cell": "no-context", "question": "Would Caroline be considered religious?", "opus_verdict": "incorrect", "opus_failure_mode": "F1", "gpt_verdict": "incorrect", "gpt_failure_mode": "F1", "model_answer": "Cannot be determined without context.", "ground_truth": "Somewhat, but not extremely religious", "context": "Session 12 (1:50 pm on 17 August, 2023):\nCaroline: Hey Mel! How're ya doin'? Recently, I had a not-so-great experience on a hike. I ran into a group of religious conservatives who said something that really upset me. It made me think how much work we still have to do for LGBTQ rights. It's been so helpful to have people around me who accept and support me, so I know I'll be ok!\n\nSession 14 (1:33 pm on 25 August, 2023):\nCaroline: Thanks! It was made for a local church and shows time changing our lives. I made it to show my own journey as a transgender woman and how we should accept growth and change. [image: a photo of a large stained glass window in a church]"}
{"instance_id": "locomo_conv-44_q000", "cell": "no-context", "question": "Which year did Audrey adopt the first three of her dogs?", "opus_verdict": "incorrect", "opus_failure_mode": "F4", "gpt_verdict": "incorrect", "gpt_failure_mode": "F4", "model_answer": "1954", "ground_truth": "2020", "context": "Session 1 (1:10 pm on 27 March, 2023):\nAudrey: I've had them for 3 years! Their names are Pepper, Precious and Panda. I can't live without my little ones!"}
{"instance_id": "locomo_conv-41_q080", "cell": "no-context", "question": "What did Maria make for her home to remind her of a trip to England?", "opus_verdict": "incorrect", "opus_failure_mode": "F4", "gpt_verdict": "incorrect", "gpt_failure_mode": "F3", "model_answer": "A wind chime", "ground_truth": "painting of a castle on a hill", "context": "Session 8 (6:03 pm on 6 March, 2023):\nMaria: Thanks, John! I got the idea from that trip to England a few years ago - I was mesmerized by the castles. I wanted to bring that same magic to my home, like a reminder of the world's beauty. I made a painting too! [image: a photography of a painting of a castle on a hill]"}
{"instance_id": "locomo_conv-50_q086", "cell": "oracle-context", "question": "What sports activity is Calvin planning to try after the tour with Frank Ocean?", "opus_verdict": "correct", "opus_failure_mode": null, "gpt_verdict": "correct", "gpt_failure_mode": null, "model_answer": "Skiing", "ground_truth": "Skiing", "context": "Session 9 (3:15 pm on 21 June, 2023):\nCalvin: Nope, not yet! I haven't been there before, but it's on my to-do list for after my tour with Frank Ocean ends. Can't wait to see them in person! Look at that snowy peak. [image: a photo of a person on skis on a snowy mountain]"}
{"instance_id": "locomo_conv-26_q059", "cell": "oracle-context", "question": "Would Caroline be considered religious?", "opus_verdict": "incorrect", "opus_failure_mode": "F3", "gpt_verdict": "incorrect", "gpt_failure_mode": "F2", "model_answer": "Yes", "ground_truth": "Somewhat, but not extremely religious", "context": "Session 12 (1:50 pm on 17 August, 2023):\nCaroline: Hey Mel! How're ya doin'? Recently, I had a not-so-great experience on a hike. I ran into a group of religious conservatives who said something that really upset me. It made me think how much work we still have to do for LGBTQ rights. It's been so helpful to have people around me who accept and support me, so I know I'll be ok!\n\nSession 14 (1:33 pm on 25 August, 2023):\nCaroline: Thanks! It was made for a local church and shows time changing our lives. I made it to show my own journey as a transgender woman and how we should accept growth and change. [image: a photo of a large stained glass window in a church]"}
{"instance_id": "locomo_conv-44_q000", "cell": "oracle-context", "question": "Which year did Audrey adopt the first three of her dogs?", "opus_verdict": "correct", "opus_failure_mode": null, "gpt_verdict": "correct", "gpt_failure_mode": null, "model_answer": "2020", "ground_truth": "2020", "context": "Session 1 (1:10 pm on 27 March, 2023):\nAudrey: I've had them for 3 years! Their names are Pepper, Precious and Panda. I can't live without my little ones!"}
{"instance_id": "locomo_conv-41_q080", "cell": "oracle-context", "question": "What did Maria make for her home to remind her of a trip to England?", "opus_verdict": "incorrect", "opus_failure_mode": "F2", "gpt_verdict": "incorrect", "gpt_failure_mode": "F2", "model_answer": "A painting", "ground_truth": "painting of a castle on a hill", "context": "Session 8 (6:03 pm on 6 March, 2023):\nMaria: Thanks, John! I got the idea from that trip to England a few years ago - I was mesmerized by the castles. I wanted to bring that same magic to my home, like a reminder of the world's beauty. I made a painting too! [image: a photography of a painting of a castle on a hill]"}
{"instance_id": "locomo_conv-50_q086", "cell": "full-context", "question": "What sports activity is Calvin planning to try after the tour with Frank Ocean?", "opus_verdict": "correct", "opus_failure_mode": null, "gpt_verdict": "correct", "gpt_failure_mode": null, "model_answer": "skiing", "ground_truth": "Skiing", "context": "Session 9 (3:15 pm on 21 June, 2023):\nCalvin: Nope, not yet! I haven't been there before, but it's on my to-do list for after my tour with Frank Ocean ends. Can't wait to see them in person! Look at that snowy peak. [image: a photo of a person on skis on a snowy mountain]"}
{"instance_id": "locomo_conv-26_q059", "cell": "full-context", "question": "Would Caroline be considered religious?", "opus_verdict": "incorrect", "opus_failure_mode": "F1", "gpt_verdict": "incorrect", "gpt_failure_mode": "F1", "model_answer": "unknown", "ground_truth": "Somewhat, but not extremely religious", "context": "Session 12 (1:50 pm on 17 August, 2023):\nCaroline: Hey Mel! How're ya doin'? Recently, I had a not-so-great experience on a hike. I ran into a group of religious conservatives who said something that really upset me. It made me think how much work we still have to do for LGBTQ rights. It's been so helpful to have people around me who accept and support me, so I know I'll be ok!\n\nSession 14 (1:33 pm on 25 August, 2023):\nCaroline: Thanks! It was made for a local church and shows time changing our lives. I made it to show my own journey as a transgender woman and how we should accept growth and change. [image: a photo of a large stained glass window in a church]"}
{"instance_id": "locomo_conv-44_q000", "cell": "full-context", "question": "Which year did Audrey adopt the first three of her dogs?", "opus_verdict": "correct", "opus_failure_mode": null, "gpt_verdict": "correct", "gpt_failure_mode": null, "model_answer": "2020", "ground_truth": "2020", "context": "Session 1 (1:10 pm on 27 March, 2023):\nAudrey: I've had them for 3 years! Their names are Pepper, Precious and Panda. I can't live without my little ones!"}
{"instance_id": "locomo_conv-41_q080", "cell": "full-context", "question": "What did Maria make for her home to remind her of a trip to England?", "opus_verdict": "incorrect", "opus_failure_mode": "F2", "gpt_verdict": "incorrect", "gpt_failure_mode": "F2", "model_answer": "a painting", "ground_truth": "painting of a castle on a hill", "context": "Session 8 (6:03 pm on 6 March, 2023):\nMaria: Thanks, John! I got the idea from that trip to England a few years ago - I was mesmerized by the castles. I wanted to bring that same magic to my home, like a reminder of the world's beauty. I made a painting too! [image: a photography of a painting of a castle on a hill]"}
{"instance_id": "locomo_conv-50_q086", "cell": "retrieval", "question": "What sports activity is Calvin planning to try after the tour with Frank Ocean?", "opus_verdict": "incorrect", "opus_failure_mode": "F1", "gpt_verdict": "incorrect", "gpt_failure_mode": "F1", "model_answer": "Not mentioned", "ground_truth": "Skiing", "context": "Session 9 (3:15 pm on 21 June, 2023):\nCalvin: Nope, not yet! I haven't been there before, but it's on my to-do list for after my tour with Frank Ocean ends. Can't wait to see them in person! Look at that snowy peak. [image: a photo of a person on skis on a snowy mountain]"}
{"instance_id": "locomo_conv-26_q059", "cell": "retrieval", "question": "Would Caroline be considered religious?", "opus_verdict": "incorrect", "opus_failure_mode": "F3", "gpt_verdict": "incorrect", "gpt_failure_mode": "F3", "model_answer": "No.", "ground_truth": "Somewhat, but not extremely religious", "context": "Session 12 (1:50 pm on 17 August, 2023):\nCaroline: Hey Mel! How're ya doin'? Recently, I had a not-so-great experience on a hike. I ran into a group of religious conservatives who said something that really upset me. It made me think how much work we still have to do for LGBTQ rights. It's been so helpful to have people around me who accept and support me, so I know I'll be ok!\n\nSession 14 (1:33 pm on 25 August, 2023):\nCaroline: Thanks! It was made for a local church and shows time changing our lives. I made it to show my own journey as a transgender woman and how we should accept growth and change. [image: a photo of a large stained glass window in a church]"}
{"instance_id": "locomo_conv-44_q000", "cell": "retrieval", "question": "Which year did Audrey adopt the first three of her dogs?", "opus_verdict": "incorrect", "opus_failure_mode": "F1", "gpt_verdict": "incorrect", "gpt_failure_mode": "F1", "model_answer": "Not mentioned.", "ground_truth": "2020", "context": "Session 1 (1:10 pm on 27 March, 2023):\nAudrey: I've had them for 3 years! Their names are Pepper, Precious and Panda. I can't live without my little ones!"}
{"instance_id": "locomo_conv-41_q080", "cell": "retrieval", "question": "What did Maria make for her home to remind her of a trip to England?", "opus_verdict": "incorrect", "opus_failure_mode": "F2", "gpt_verdict": "incorrect", "gpt_failure_mode": "F2", "model_answer": "A painting.", "ground_truth": "painting of a castle on a hill", "context": "Session 8 (6:03 pm on 6 March, 2023):\nMaria: Thanks, John! I got the idea from that trip to England a few years ago - I was mesmerized by the castles. I wanted to bring that same magic to my home, like a reminder of the world's beauty. I made a painting too! [image: a photography of a painting of a castle on a hill]"}
{"instance_id": "locomo_conv-50_q086", "cell": "agentic", "question": "What sports activity is Calvin planning to try after the tour with Frank Ocean?", "opus_verdict": "incorrect", "opus_failure_mode": "F1", "gpt_verdict": "incorrect", "gpt_failure_mode": "F1", "model_answer": "Max tool turns reached (3 turns, 3 tools used).", "ground_truth": "Skiing", "context": "Session 9 (3:15 pm on 21 June, 2023):\nCalvin: Nope, not yet! I haven't been there before, but it's on my to-do list for after my tour with Frank Ocean ends. Can't wait to see them in person! Look at that snowy peak. [image: a photo of a person on skis on a snowy mountain]"}
{"instance_id": "locomo_conv-26_q059", "cell": "agentic", "question": "Would Caroline be considered religious?", "opus_verdict": "incorrect", "opus_failure_mode": "F3", "gpt_verdict": "incorrect", "gpt_failure_mode": "F4", "model_answer": "Yes, Caroline has a connection to faith through her heritage (a necklace from Sweden representing \"love, faith and strength\") and by having made a painting for a local church.", "ground_truth": "Somewhat, but not extremely religious", "context": "Session 12 (1:50 pm on 17 August, 2023):\nCaroline: Hey Mel! How're ya doin'? Recently, I had a not-so-great experience on a hike. I ran into a group of religious conservatives who said something that really upset me. It made me think how much work we still have to do for LGBTQ rights. It's been so helpful to have people around me who accept and support me, so I know I'll be ok!\n\nSession 14 (1:33 pm on 25 August, 2023):\nCaroline: Thanks! It was made for a local church and shows time changing our lives. I made it to show my own journey as a transgender woman and how we should accept growth and change. [image: a photo of a large stained glass window in a church]"}
{"instance_id": "locomo_conv-44_q000", "cell": "agentic", "question": "Which year did Audrey adopt the first three of her dogs?", "opus_verdict": "incorrect", "opus_failure_mode": "F1", "gpt_verdict": "incorrect", "gpt_failure_mode": "F1", "model_answer": "Max tool turns reached (3 turns, 3 tools used).", "ground_truth": "2020", "context": "Session 1 (1:10 pm on 27 March, 2023):\nAudrey: I've had them for 3 years! Their names are Pepper, Precious and Panda. I can't live without my little ones!"}
{"instance_id": "locomo_conv-41_q080", "cell": "agentic", "question": "What did Maria make for her home to remind her of a trip to England?", "opus_verdict": "incorrect", "opus_failure_mode": "F2", "gpt_verdict": "incorrect", "gpt_failure_mode": "F2", "model_answer": "a painting", "ground_truth": "painting of a castle on a hill", "context": "Session 8 (6:03 pm on 6 March, 2023):\nMaria: Thanks, John! I got the idea from that trip to England a few years ago - I was mesmerized by the castles. I wanted to bring that same magic to my home, like a reminder of the world's beauty. I made a painting too! [image: a photography of a painting of a castle on a hill]"}