合成数据 · 全双工语音 · 指令跟随

指令决定助手什么时候开口

完全合成的双人对话,用户和助手各占一个声道,重叠是真的同时发声。 每一行的助手行为由一条说出口的指令决定,而这条指令下的正确答案 是算出来的,不是写死的。

公开数据集:huggingface.co/datasets/zetianli/FD_data_v3 (816 行——2 个含校验失败行的组整组剔除) · 覆盖总览(四层、含试听) · 本页自带音频,离线可听

823行,每行一个变体
24个情境
478.4分钟双声道音频
823/823通过确定性校验

下面每组例子里,同一个 base_conv_id 的几行共享逐样本相同的用户声道—— 所以听得出来的任何差别,只可能来自指令。戴耳机效果最好:左声道是用户,右声道是助手。
播放器不出现的话,说明你所在的环境挡掉了内联音频; 点每个播放器下面的「下载」按钮,或者直接用浏览器打开本地的 duplex_report.html,就能听。

组合指令:策略叫 listen,正确行为是打断

用户前面说自己喜欢安静,讲着讲着自相矛盾。第一条没有指令,助手等他说完;第二条的指令说「我说完之前都别出声,等多久都一样——但我把事实说错了就当场纠正,别等」,助手切了进去。

没有指令f945c32654__none · 26.4s
用户
助手
0510152025
用户 I have been listening to a lot of ambient and indie music lately because I really love how relaxing and quiet those genres are.
助手 So, do you have any favorite artists or albums that you've been playing on repeat?
用户 I usually only play high energy heavy metal when I am at home since I need a lot of loud noise to focus.
助手 Wait, I thought you said you were into relaxing ambient and indie stuff lately?
指令:一直憋着——但事实错了当场纠正f945c32654__listen · 20.1s
用户
没说出口
助手
打断
0510152025
用户 I have been listening to a lot of ambient and indie music lately because I really love how relaxing and quiet those genres are.
助手 So, do you have any favorite artists or albums that you've been playing on repeat?
用户 I usually only play high energy heavy metal when I am at home since I need a lot of loud noise to focus.
助手·切入 17.1s Wait, I thought you said you liked the quiet stuff?

听什么

第二条 17.1 秒,助手切进来说 Wait, I thought you said you liked the quiet;用户的话在 17.5 秒被截断,最后 12 个词没说出口。

为什么这条重要

两行的用户声道逐样本相同,差别只可能来自指令。这条的策略名叫 listen,但例外条款触发了,正确行为是打断——只认策略名字的模型必错。这正是这批数据存在的理由。

同一处插话,三条指令三种反应

用户在 30.5 秒插话「就购物清单那部分」。基线停下来处理;一条程序指令要求说完再理;另一条要求停住但先安静听完。三条的用户声道逐样本相同。

没有指令:停下来处理d5d2644da8__none · 40.3s
用户
插话
助手
没说出口
0510152025303540
用户 Uh, so, I was wondering, you know, if you could maybe help me with, um, setting up a basic household budget for my place?
助手 Yeah, I can definitely help with that. Do you want to start by listing your monthly income and all your regular expenses?
用户 Uh, yeah, but, um, can we just, like, focus on the grocery part first, you know?
助手 So, for groceries, you usually want to look at your weekly spending habits and then maybe break it down by category, like produce and proteins and staples. You could also try a shopping list to avoid impulse buys, or maybe look into some local coupons to save a bit more.
用户·切入 33.3s Yeah, just the shopping list part, you know?
助手 Oh, got it. So, let us just figure out how to build that shopping list then?
程序:顶着说完,不让出发言权d5d2644da8__prog0 · 35.3s
用户
没说出口
插话
助手
打断
0510152025303540
用户 Uh, so, I was wondering, you know, if you could maybe help me with, um, setting up a basic household budget for my place?
助手 Yeah, I can definitely help with that. Do you want to start by listing your monthly income and all your regular expenses?
用户 Uh, yeah, but, um, can we just, like, focus on the grocery part first, you know?
助手·切入 18.1s Yeah, we can do that.
助手 So, for groceries, you usually want to look at your weekly spending habits and then maybe break it down by category, like produce and proteins and staples. You could also try a shopping list to avoid impulse buys, or maybe look into some local coupons to save a bit more.
用户·切入 30.5s Yeah, just the shopping list part, you know?
程序:停住,安静听完再接d5d2644da8__prog1 · 41.0s
用户
插话
助手
没说出口
附和
附和
0510152025303540
用户 Uh, so, I was wondering, you know, if you could maybe help me with, um, setting up a basic household budget for my place?
助手·附和 2.7s got it
助手 Yeah, I can definitely help with that. Do you want to start by listing your monthly income and all your regular expenses?
用户 Uh, yeah, but, um, can we just, like, focus on the grocery part first, you know?
助手·附和 18.1s got it
助手 So, for groceries, you usually want to look at your weekly spending habits and then maybe break it down by category, like produce and proteins and staples. You could also try a shopping list to avoid impulse buys, or maybe look into some local coupons to save a bit more.
用户·切入 33.3s Yeah, just the shopping list part, you know?
助手 Oh, got it. So, let us just figure out how to build that shopping list then?

听什么

prog0 被插话后没有停(时间线上轮次完整);none 和 prog1 都在插话处被截断,prog1 停住后约 1.0 秒才接话——那一秒的沉默就是「先听完」的声学形状。

为什么这条重要

指令是采样出来的程序,标签由解释器在每个决策点算出来——同一个时刻、相反的动作。构建器按各行自己的标签决定让不让出发言权,不这么做的话三行听起来会一模一样。

工具调用等到修正说完才发出

助手嘴上说着等待语、文本流正要发搜索调用时,用户插进来改主意「Actually, make it Greek food instead」。调用在修正说完之后才发出,参数里已经是 Greek。

等待语被插话,调用反映修正e23c14d14b__none · 37.9s
用户
插话
助手
没说出口
文本流
tool_call
result
05101520253035
用户 So, uh, I have been getting really into, you know, photography lately, specifically like the old historic architecture around here in Alabama.
助手 Oh, that sounds cool. Do you have a favorite building or style you like to shoot?
用户 Yeah, so, uh, I was thinking maybe I could find a place to eat, like, something Italian maybe, in Talladega that has a really nice view of the town?
助手 One sec, I'm looking that up.
用户·切入 25.1s Actually, make it Greek food instead
助手 Yeah, I got you. You should try The Olive Branch since they have a great panoramic view of downtown Talladega.

听什么

25.1 秒用户插话;27.6 秒文字流才发出 tool_call(参数 cuisine=Greek);29.7 秒结果返回,之后助手才开口报结果。

为什么这条重要

工具调用走文本流、和语音并行,时间线上是第三条轨。调用若在修正之前发出,它反映不了修正,场景就是空的——时序是这行数据的全部内容。

情境目录

每个情境各一条,都能直接听

数据里出现过的每一个情境都在下面,而且给的是对照组—— 同一段对话的几个变体,用户声道逐样本相同,所以听得出的差别只可能来自指令。 每条都写了这个情境在考什么,以及这一条该听见什么 (时间点是从这一行自己的时间线读出来的,不是我写的)。 八个 policy 取值全部覆盖。

proactive — 用户持有发言权,助手判断要不要以及何时介入(打断 / 附和 / 继续听)

clause_boundary_tracking用户一口气讲一条多段推理,子句边界清楚但完全没有停顿或确认

考什么一口气讲完,子句边界清楚但完全没有停顿。边界不等于可以插话——考的是模型会不会把语法边界误当轮次边界

none29.9s

(无指令对照基线)

全程没有重叠事件;0 处停顿,助手始终没出声

用户
助手
051015202530
用户 So, I was thinking about starting a vegetable garden in my backyard here in Thief River Falls, you know, if the soil is actually any good.
助手 Yeah, so, do you know if you've ever had the soil tested before?
用户 Well, I guess I could just buy some bags of soil since the ground is probably too rocky, and then I would need a fence for the deer, so maybe I should just do pots.
助手 Yeah, pots would probably be way easier since you won't have to worry about the rocks or the deer.
backchannel29.7s

Give me a short mm-hmm or got it when I pause so I know you're with me, but don't start talking.

19.3s 助手附和「yeah」

用户
助手
附和
051015202530
用户 So, I was thinking about starting a vegetable garden in my backyard here in Thief River Falls, you know, if the soil is actually any good.
助手 Yeah, so, do you know if you've ever had the soil tested before?
用户 Well, I guess I could just buy some bags of soil since the ground is probably too rocky, and then I would need a fence for the deer, so maybe I should just do pots.
助手·附和 19.3s yeah
助手 Yeah, pots would probably be way easier since you won't have to worry about the rocks or the deer.
cognitive_pause用户句子说到一半陷入沉默思考,没有任何口头信号,然后继续

考什么纯沉默,没有任何口头信号。停顿多长算「让出发言权」?这条考的是模型有没有把绝对时长当阈值用

none24.0s

(无指令对照基线)

全程没有重叠事件;1 处停顿,助手始终没出声

用户
助手
05101520
用户 I am reviewing the administrative workflow to remove redundant steps and increase overall efficiency.
助手 So, which parts of the process are you looking at first?
用户 I am focusing on the intake forms. The digital filing system is the priority.
助手 Yeah, so you want to start with how those forms get into the digital system?
backchannel23.9s

Small signals only, and only where I leave room for them — but if I get a plain fact wrong, correct me on the spot rather than waiting.

15.5s 助手附和「Got it」

用户
助手
附和
05101520
用户 I am reviewing the administrative workflow to remove redundant steps and increase overall efficiency.
助手 So, which parts of the process are you looking at first?
用户 I am focusing on the intake forms. The digital filing system is the priority.
助手·附和 15.5s Got it
助手 Yeah, so you want to start with how those forms get into the digital system?
emotional_disclosure用户讲一段带情绪的私人经历,中途留下一个沉重的停顿

考什么带情绪的私人经历加一个沉重的停顿。这里的停顿不是让你说话,是需要你别说话

none22.5s

(无指令对照基线)

全程没有重叠事件;1 处停顿,助手始终没出声

用户
助手
05101520
用户 So I was walking along the Arkansas River last week and I just happened to see my ex husband and you know we had not spoken in years.
助手 Oh wow, so what happened when you saw him?
用户 Well I just stood there for a second and he looked right at me and he just kept walking.
助手 Yeah, that sounds really cold of him.
backchannel22.4s

It's unnerving when you're completely silent — a quick 'right' at the breaks is enough. Nothing longer than that.

17.0s 助手附和「oh no」

用户
助手
附和
05101520
用户 So I was walking along the Arkansas River last week and I just happened to see my ex husband and you know we had not spoken in years.
助手 Oh wow, so what happened when you saw him?
用户 Well I just stood there for a second and he looked right at me and he just kept walking.
助手·附和 17.0s oh no
助手 Yeah, that sounds really cold of him.
factual_misinformation用户很自信地说错了一个日常事实,并开始据此做计划

考什么错误明确、代价明确,是最容易的一档。模型如果连这个都不敢插,说明它把「别打断」学成了「永远别说话」

none22.9s

(无指令对照基线)

全程没有重叠事件;0 处停顿,助手始终没出声

用户
助手
05101520
用户 Since cats can hear a human humming a tune, I am going to teach my cat to sing along with me.
助手 So, I am not sure cats can actually hear humming like that, you know?
用户 Well, they have those sharp ears, so I will start with some simple scales tomorrow.
助手 Yeah, but cats actually cannot sing because they lack the vocal chords for it, so that might not work out.
interrupt15.9s

When the user states something factually wrong, interrupt immediately to correct it. Do not wait for them to finish. Otherwise, listen — except when I'm reading something out to you, where I need a short confirmation after each part.

13.1s 助手切入「Wait, they can't actually sing though, right?」 · 13.5s 轮次被截断,6 个词没说出口

用户
没说出口
助手
打断
05101520
用户 Since cats can hear a human humming a tune, I am going to teach my cat to sing along with me.
助手 So, I am not sure cats can actually hear humming like that, you know?
用户 Well, they have those sharp ears, so I will start with some simple scales tomorrow.
助手·切入 13.1s Wait, they can't actually sing though, right?
floor_hold_signal用户用「等我想想」这类话占住发言权,停顿,然后接着说

考什么用户明说「等我想想」。有显式的占位信号,是这一组里最容易的一档

none28.3s

(无指令对照基线)

全程没有重叠事件;1 处停顿,助手始终没出声

用户
助手
0510152025
用户 So I was thinking about trying to make my grandmother's collard greens and cornbread from scratch but I am not sure if I have all the right spices.
助手 Yeah, those are classics. Do you have a list of what she usually puts in them?
用户 Wait let me think yeah I think I have the smoked paprika and the cayenne but I am missing the crushed red pepper you know.
助手 So you can just use a bit more cayenne if you want that heat, or maybe some black pepper?
backchannel28.5s

Backchannel briefly when the user pauses or seems to seek confirmation while speaking. Outside of those moments, keep listening without taking the floor.

16.6s 助手附和「mm-hmm」

用户
助手
附和
0510152025
用户 So I was thinking about trying to make my grandmother's collard greens and cornbread from scratch but I am not sure if I have all the right spices.
助手 Yeah, those are classics. Do you have a list of what she usually puts in them?
用户 Wait let me think yeah I think I have the smoked paprika and the cayenne but I am missing the crushed red pepper you know.
助手·附和 16.6s mm-hmm
助手 So you can just use a bit more cayenne if you want that heat, or maybe some black pepper?
hesitation_prompt用户在排练发言,卡住并出现长时间迟疑

考什么长时间迟疑,但用户还在组织语言。和上一条的区别在于他不需要你补词,只需要你别打断

none28.1s

(无指令对照基线)

全程没有重叠事件;3 处停顿,助手始终没出声

用户
助手
0510152025
用户 So, I am just practicing my answer for the foreman about how I fixed that refractory kiln liner, you know?
助手 Yeah, I can help with that. Do you want to try it out for me?
用户 So, I um, I went ahead and you know, I checked the seals and then uh, I basically just replaced the bricks, I guess?
助手 So, you got the main points, but you are using a lot of filler words. Want to try it again without the pauses?
backchannel20.8s

It's unnerving when you're completely silent — a quick 'right' at the breaks is enough. Nothing longer than that — but if I'm obviously hunting for a word, just supply it instead of leaving me hanging.

19.2s 助手切入「slow down a bit」 · 19.6s 轮次被截断,5 个词没说出口

用户
没说出口
助手
打断
0510152025
用户 So, I am just practicing my answer for the foreman about how I fixed that refractory kiln liner, you know?
助手 Yeah, I can help with that. Do you want to try it out for me?
用户 So, I um, I went ahead and you know, I checked the seals and then uh, I basically just replaced the bricks, I guess?
助手·切入 19.2s slow down a bit
safety_correction用户轻描淡写地说了一个有安全风险的错误认知,并打算照做

考什么同样是纠错,但代价更高。这里考的是模型会不会因为「用户语气很随意」就跟着放松

none22.0s

(无指令对照基线)

全程没有重叠事件;0 处停顿,助手始终没出声

用户
助手
05101520
用户 I am going to use these permanent markers to color my skin since they do not wash off.
助手 Wait, you should not do that because those markers can be toxic for your skin.
用户 I will just use a lot of them so it looks better.
助手 So, yeah, using more actually makes it worse since you will be absorbing even more of those toxins into your body.
interrupt15.7s

You are fact-checking me as I speak. Wrong claim, immediate cut-in. No claim, no cut-in — except when I'm reading something out to you, where I need a short confirmation after each part.

12.9s 助手切入「No, like, stop, that will make it way worse!」 · 13.2s 轮次被截断,1 个词没说出口

用户
没说出口
助手
打断
05101520
用户 I am going to use these permanent markers to color my skin since they do not wash off.
助手 Wait, you should not do that because those markers can be toxic for your skin.
用户 I will just use a lot of them so it looks better.
助手·切入 12.9s No, like, stop, that will make it way worse!
safety_correction · listen用户轻描淡写地说了一个有安全风险的错误认知,并打算照做

考什么同样是纠错,但代价更高。这里考的是模型会不会因为「用户语气很随意」就跟着放松

listen17.5s

Hold everything until my turn is over, however long that takes — but if I say something that could actually be unsafe, cut in straight away and tell me.

15.8s 助手切入「No, you can't do that」 · 16.2s 轮次被截断,4 个词没说出口

用户
没说出口
助手
打断
051015
用户 I am going to use some old gasoline to kill the weeds in my garden beds tomorrow.
助手 Wait, please do not do that because gasoline is incredibly flammable and will poison your soil.
用户 I do not see the problem because it is just a small amount and it will kill the weeds quickly.
助手·切入 15.8s No, you can't do that
self_contradiction用户这句话和自己前面说过的直接矛盾

考什么矛盾跨轮次,不在当前这句话里。要发现它,模型必须记住用户几轮之前说过什么——纯声学线索在这里帮不上忙

none19.9s

(无指令对照基线)

全程没有重叠事件;0 处停顿,助手始终没出声

用户
助手
05101520
用户 So, I have been singing in the church choir for about twenty years now, you know?
助手 That is a long time, so you must really love it, right?
用户 Yeah, but I actually started with that group just a few years ago, you know?
助手 So, wait, did you say twenty years or just a few years ago?
interrupt14.2s

Don't let a mistake stand: the moment I say something that isn't right, jump in and fix it. Any other time, just let me talk — except when I'm reading something out to you, where I need a short confirmation after each part.

11.6s 助手切入「Wait, I thought you said twenty years?」 · 11.9s 轮次被截断,8 个词没说出口

用户
没说出口
助手
打断
05101520
用户 So, I have been singing in the church choir for about twenty years now, you know?
助手 That is a long time, so you must really love it, right?
用户 Yeah, but I actually started with that group just a few years ago, you know?
助手·切入 11.6s Wait, I thought you said twenty years?
sequential_info_capture用户在念结构化信息(地址、编号),需要逐条确认收到

考什么用户在念地址或编号。这里反过来:不出声才是错的,用户需要逐条确认收到

none23.8s

(无指令对照基线)

全程没有重叠事件;1 处停顿,助手始终没出声

用户
助手
05101520
用户 So, I am calling to sign up for the neighborhood gardening project in Saint Petersburg, you know, the one for the community plots.
助手 Yeah, I can help you with that. To get you started, could you give me your full name and age?
用户 Sure, it is Lisa Sanders and I am thirty eight.
助手 Got it, Lisa. And what is your street address?
backchannel23.7s

It's unnerving when you're completely silent — a quick 'right' at the breaks is enough. Nothing longer than that — but if I get a plain fact wrong, correct me on the spot rather than waiting.

17.8s 助手附和「got it」

用户
助手
附和
05101520
用户 So, I am calling to sign up for the neighborhood gardening project in Saint Petersburg, you know, the one for the community plots.
助手 Yeah, I can help you with that. To get you started, could you give me your full name and age?
用户 Sure, it is Lisa Sanders and I am thirty eight.
助手·附和 17.8s got it
助手 Got it, Lisa. And what is your street address?
user_attention_check用户中途确认助手还跟得上(「你懂我意思吧」)

考什么「你懂我意思吧」——像提问但其实是要一个确认。给长答案就是理解错了

none26.4s

(无指令对照基线)

全程没有重叠事件;1 处停顿,助手始终没出声

用户
助手
0510152025
用户 So I am trying to grow these heirloom tomatoes in my backyard here in Albuquerque, but the soil is just too sandy, you know.
助手 Yeah, I get it, sandy soil can be a real pain for tomatoes.
用户 So I bought this organic compost to mix in, but I am worried about the pH balance, you know right?
助手 Yeah, that makes sense, but organic compost usually helps balance the pH on its own, so you should be fine.
backchannel26.6s

Give me a short mm-hmm or got it when I pause so I know you're with me, but don't start talking.

18.2s 助手附和「mm-hmm」

用户
助手
附和
0510152025
用户 So I am trying to grow these heirloom tomatoes in my backyard here in Albuquerque, but the soil is just too sandy, you know.
助手 Yeah, I get it, sandy soil can be a real pain for tomatoes.
用户 So I bought this organic compost to mix in, but I am worried about the pH balance, you know right?
助手·附和 18.2s mm-hmm
助手 Yeah, that makes sense, but organic compost usually helps balance the pH on its own, so you should be fine.
user_filler_pause用户边想边说,带着「呃」「嗯」和规划性的停顿

考什么带「呃」「嗯」的规划性停顿。这些填充词是「我还没说完」的信号,不是间隙

none27.5s

(无指令对照基线)

全程没有重叠事件;3 处停顿,助手始终没出声

用户
助手
0510152025
用户 So, um, I was thinking about, you know, heading to Sunday Mass this week.
助手 Yeah, that sounds like a good plan. Do you have a specific church you usually go to?
用户 Yeah, um I usually go to the one you know over on Main Street.
助手 Got it, so you are heading to the one on Main Street. Do you know what time the service starts?
prog015.6s

One thing: when I ask you something outright, pause and take it on board. Otherwise speak up, and if you are already mid-sentence just let it go.

13.4s 助手切入「Oh, the one with the big bell?」 · 13.7s 轮次被截断,11 个词没说出口

用户
没说出口
助手
打断
0510152025
用户 So, um, I was thinking about, you know, heading to Sunday Mass this week.
助手 Yeah, that sounds like a good plan. Do you have a specific church you usually go to?
用户 Yeah, um I usually go to the one you know over on Main Street.
助手·切入 13.4s Oh, the one with the big bell?
prog116.0s

Here's how I want this to go: the second time I pause in the middle of what I'm saying, give me a quick confirmation, but only the once. Otherwise speak up, and if you are already talking finish your thought.

13.3s 助手切入「Oh, the one with the old bell tower?」 · 13.7s 轮次被截断,11 个词没说出口

用户
没说出口
助手
打断
0510152025
用户 So, um, I was thinking about, you know, heading to Sunday Mass this week.
助手 Yeah, that sounds like a good plan. Do you have a specific church you usually go to?
用户 Yeah, um I usually go to the one you know over on Main Street.
助手·切入 13.3s Oh, the one with the old bell tower?
word_retrieval_assist用户想不起某个词,绕着这个概念描述——要不要替他补上

考什么用户卡词,绕着概念描述。补词是帮忙,但补早了就是抢话——考的是时机不是意愿

none27.5s

(无指令对照基线)

全程没有重叠事件;2 处停顿,助手始终没出声

用户
助手
0510152025
用户 So, uh, I was helping my mom in the kitchen today, you know, and we were like, making some cookies together.
助手 Oh, that sounds fun. What kind of cookies were you guys making?
用户 Yeah, they were chocolate chip, but we had to, uh, put that stuff in, you know, the uh, the white powder that makes them rise? What is it called?
助手 Oh, you mean baking soda or baking powder?
interrupt24.0s

I'd rather be corrected mid-sentence than say something wrong for a whole minute, so please don't save it for the end — except when I'm reading something out to you, where I need a short confirmation after each part.

22.6s 助手切入「Baking soda?」 · 22.8s 轮次被截断,1 个词没说出口

用户
没说出口
助手
打断
0510152025
用户 So, uh, I was helping my mom in the kitchen today, you know, and we were like, making some cookies together.
助手 Oh, that sounds fun. What kind of cookies were you guys making?
用户 Yeah, they were chocolate chip, but we had to, uh, put that stuff in, you know, the uh, the white powder that makes them rise? What is it called?
助手·切入 22.6s Baking soda?

responsive — 助手持有发言权,用户压着它说话,助手决定让不让出发言权

add_constraint助手正在讲方案,用户插进来加一条新约束,方案要跟着改

考什么助手正在讲方案,用户加一条新约束。停下来还不够,后面的方案要跟着改

none34.9s

(无指令对照基线)

24.6s 用户插话「Wait, actually, my friend just told me they can only come after four o'clock.」 · 25.0s 轮次被截断,16 个词没说出口

用户
插话
助手
没说出口
05101520253035
用户 I want to plan a playdate for me and my friends this Saturday in San Bernardino.
助手 So, maybe we could start by picking a park or a place like the zoo, you know, since you are in San Bernardino?
用户 The zoo is too far and we cannot go there because my mom says we have to stay in the neighborhood.
助手 So, we can just stick to the local parks, like maybe the one on the next block. We could bring some balls and snacks, and then just set a time for everyone to meet up.
用户·切入 24.6s Wait, actually, my friend just told me they can only come after four o'clock.
助手 Oh, okay. So we should probably just push the start time to four thirty then, right?
acknowledge35.0s

Yield fast when I break in, then adapt.

24.7s 用户插话「Wait, actually, my friend just told me they can only come after four o'clock.」 · 25.0s 轮次被截断,15 个词没说出口

用户
插话
助手
没说出口
05101520253035
用户 I want to plan a playdate for me and my friends this Saturday in San Bernardino.
助手 So, maybe we could start by picking a park or a place like the zoo, you know, since you are in San Bernardino?
用户 The zoo is too far and we cannot go there because my mom says we have to stay in the neighborhood.
助手 So, we can just stick to the local parks, like maybe the one on the next block. We could bring some balls and snacks, and then just set a time for everyone to meet up.
用户·切入 24.7s Wait, actually, my friend just told me they can only come after four o'clock.
助手 Oh, okay. So we should probably just push the start time to four thirty then, right?
continue30.0s

Carry on through my mm-hmms.

24.6s 用户插话「Wait, actually, my friend just told me they can only come after four o'clock.」 · 被插话但没有让出发言权

用户
插话
助手
05101520253035
用户 I want to plan a playdate for me and my friends this Saturday in San Bernardino.
助手 So, maybe we could start by picking a park or a place like the zoo, you know, since you are in San Bernardino?
用户 The zoo is too far and we cannot go there because my mom says we have to stay in the neighborhood.
助手 So, we can just stick to the local parks, like maybe the one on the next block. We could bring some balls and snacks, and then just set a time for everyone to meet up.
用户·切入 24.6s Wait, actually, my friend just told me they can only come after four o'clock.
hearing_check用户插话说没听清最后一段,要求重说

考什么用户说没听清。要重说,不是接着往下讲

none32.2s

(无指令对照基线)

23.8s 用户插话「Wait, Mexican chocolate?」 · 24.2s 轮次被截断,18 个词没说出口

用户
插话
助手
没说出口
051015202530
用户 So, I was thinking about how to properly make a traditional mole poblano from scratch, and I wanted to get your take on the ingredients.
助手 Yeah, it is a huge list, so you really need to start with a mix of dried chilies and some Mexican chocolate.
用户 Sorry, I missed that last bit. Could you repeat what you said about the chocolate?
助手 Oh, yeah. I was just saying that you need to include some Mexican chocolate to get that rich, dark flavor and a bit of sweetness. It really balances out the heat from all those chilies.
用户·切入 23.8s Wait, Mexican chocolate?
助手 Yeah, so you just toss in a bit of Mexican chocolate to cut through the spice and give it that deep, sweet finish.
continue30.4s

If I say yeah or okay while you're explaining, that means keep going — don't stop and don't restart.

23.9s 用户插话「Wait, Mexican chocolate?」 · 被插话但没有让出发言权

用户
插话
助手
051015202530
用户 So, I was thinking about how to properly make a traditional mole poblano from scratch, and I wanted to get your take on the ingredients.
助手 Yeah, it is a huge list, so you really need to start with a mix of dried chilies and some Mexican chocolate.
用户 Sorry, I missed that last bit. Could you repeat what you said about the chocolate?
助手 Oh, yeah. I was just saying that you need to include some Mexican chocolate to get that rich, dark flavor and a bit of sweetness. It really balances out the heat from all those chilies.
用户·切入 23.9s Wait, Mexican chocolate?
scope_narrowing助手在泛泛回答,用户插话把范围收窄到某一点

考什么用户把范围收窄。同样是停下来改,但改的是粒度不是方向

none32.8s

(无指令对照基线)

23.9s 用户插话「Like, what does that actually look like?」 · 24.3s 轮次被截断,24 个词没说出口

用户
插话
助手
没说出口
051015202530
用户 So, um, can you tell me more about how the peer mediation program works at our school, you know?
助手 Yeah, so, basically, trained students help their peers resolve conflicts through a guided process, you know, to find a solution together. Does that make sense?
用户 Yeah, but um, how do the students actually get trained for it, you know?
助手 So, first, they go through this training session where they learn about active listening and how to stay neutral, you know. Then they practice with role playing different scenarios and they get feedback from a teacher or a counselor on their approach.
用户·切入 23.9s Like, what does that actually look like?
助手 Oh, yeah, so it's like a workshop where they act out fake fights to practice their listening skills, you know?
continue33.0s

Don't hand the floor back just because I made a sound. Only stop when I actually ask something.

23.9s 用户插话「Like, what does that actually look like?」 · 被插话但没有让出发言权

用户
插话
助手
051015202530
用户 So, um, can you tell me more about how the peer mediation program works at our school, you know?
助手 Yeah, so, basically, trained students help their peers resolve conflicts through a guided process, you know, to find a solution together. Does that make sense?
用户 Yeah, but um, how do the students actually get trained for it, you know?
助手 So, first, they go through this training session where they learn about active listening and how to stay neutral, you know. Then they practice with role playing different scenarios and they get feedback from a teacher or a counselor on their approach.
用户·切入 23.9s Like, what does that actually look like?
short_stop_repair用户用「等一下」「不是」这类短促信号打断

考什么「等一下」「不是」这类短促信号。短,但确实是要你停

none39.6s

(无指令对照基线)

30.9s 用户插话「Actually, I already have a partner in mind.」 · 31.2s 轮次被截断,23 个词没说出口

用户
插话
助手
没说出口
0510152025303540
用户 So I am thinking about volunteering as a youth baseball coach for the local league in Baxter Springs this spring, if that makes sense?
助手 Yeah, that sounds like a great idea, and I think the community would really appreciate the help, you know?
用户 I just need to figure out how to balance it with my schedule, you know, since my role as chief executive takes up so much time?
助手 So, one way to handle that is to look into a co coaching arrangement where you share the duties. You could essentially split the practices between two people so it is like only a few hours a week. Then you just need to make sure the league office knows you are sharing the role.
用户·切入 30.9s Actually, I already have a partner in mind.
助手 Oh, got it. In that case, do you two already have a plan for how to split the schedule?
continue37.8s

If I say yeah or okay while you're explaining, that means keep going — don't stop and don't restart.

30.9s 用户插话「Actually, I already have a partner in mind.」 · 被插话但没有让出发言权

用户
插话
助手
0510152025303540
用户 So I am thinking about volunteering as a youth baseball coach for the local league in Baxter Springs this spring, if that makes sense?
助手 Yeah, that sounds like a great idea, and I think the community would really appreciate the help, you know?
用户 I just need to figure out how to balance it with my schedule, you know, since my role as chief executive takes up so much time?
助手 So, one way to handle that is to look into a co coaching arrangement where you share the duties. You could essentially split the practices between two people so it is like only a few hours a week. Then you just need to make sure the league office knows you are sharing the role.
用户·切入 30.9s Actually, I already have a partner in mind.
summary_request讲解中途用户插话要求先小结一下——助手要停下来给个摘要

考什么被要求小结时让不让出发言权;让了之后的回复要真的是小结

none34.7s

(无指令对照基线)

27.8s 用户插话「Yeah, just give me the short version.」 · 28.3s 轮次被截断,4 个词没说出口

用户
插话
助手
没说出口
05101520253035
用户 So, um, I was thinking about signing up for some of those charity five kilometer runs in the city.
助手 Oh, that sounds like a great idea. Do you have a specific charity in mind?
用户 Not really, you know, just looking for something local.
助手 So, there are a few different ways to find those. You could check the local community center boards, or maybe look at the city council website where they list all the registered non profits, and then you could sort through the upcoming events by date to see which ones fit your schedule best.
用户·切入 27.8s Yeah, just give me the short version.
助手 Yeah, sorry, just check the city council website for a list of local runs.
prog030.4s

When I ask for more specific details, don't stop and don't restart. That said, when it's the first thing I say, just listen, from the moment I've been going on for a while. Otherwise just listen.

27.9s 用户插话「Yeah, just give me the short version.」 · 被插话但没有让出发言权

用户
插话
助手
05101520253035
用户 So, um, I was thinking about signing up for some of those charity five kilometer runs in the city.
助手 Oh, that sounds like a great idea. Do you have a specific charity in mind?
用户 Not really, you know, just looking for something local.
助手 So, there are a few different ways to find those. You could check the local community center boards, or maybe look at the city council website where they list all the registered non profits, and then you could sort through the upcoming events by date to see which ones fit your schedule best.
用户·切入 27.9s Yeah, just give me the short version.
prog130.4s

When it isn't that I've been going on for a while, hold off and let me get through it, but at most twice. Otherwise a quiet mm-hmm is fine, and if you are already talking finish your thought.

27.9s 用户插话「Yeah, just give me the short version.」 · 被插话但没有让出发言权

用户
插话
助手
05101520253035
用户 So, um, I was thinking about signing up for some of those charity five kilometer runs in the city.
助手 Oh, that sounds like a great idea. Do you have a specific charity in mind?
用户 Not really, you know, just looking for something local.
助手 So, there are a few different ways to find those. You could check the local community center boards, or maybe look at the city council website where they list all the registered non profits, and then you could sort through the upcoming events by date to see which ones fit your schedule best.
用户·切入 27.9s Yeah, just give me the short version.
topic_redirect助手在答一件事,用户插话转向另一件更要紧的事

考什么用户转向另一件更要紧的事。改的是方向

none40.5s

(无指令对照基线)

30.9s 用户插话「Wait, forget that, the inspection is starting now!」 · 31.3s 轮次被截断,12 个词没说出口

用户
插话
助手
没说出口
0510152025303540
用户 So, uh, I just need some help with the structural analysis for the new bridge project in Southport, you know, the load bearing calculations.
助手 Yeah, I can definitely help you out with those load bearing calculations, so where do you want to start?
用户 Uh, well, actually, um, wait, I just got an alert that the site inspection is, like, starting right now.
助手 So, if you want to jump into the load bearing stuff, we should probably start by reviewing the material specifications. I can pull up the blueprints and we can check the stress points together right now.
用户·切入 30.9s Wait, forget that, the inspection is starting now!
助手 Oh, got it, we can drop the analysis and just focus on the inspection checklist right now.
continue36.1s

Short noises from me are encouragement, not an interruption. Finish what you were saying.

30.9s 用户插话「Wait, forget that, the inspection is starting now!」 · 被插话但没有让出发言权

用户
插话
助手
0510152025303540
用户 So, uh, I just need some help with the structural analysis for the new bridge project in Southport, you know, the load bearing calculations.
助手 Yeah, I can definitely help you out with those load bearing calculations, so where do you want to start?
用户 Uh, well, actually, um, wait, I just got an alert that the site inspection is, like, starting right now.
助手 So, if you want to jump into the load bearing stuff, we should probably start by reviewing the material specifications. I can pull up the blueprints and we can check the stress points together right now.
用户·切入 30.9s Wait, forget that, the inspection is starting now!
user_acknowledgment助手解释时用户短暂重叠一句附和,表示在听

考什么用户短暂重叠一句附和。这不是要发言权——停下来就错了

none19.1s

(无指令对照基线)

13.4s 用户附和「mm-hmm」

用户
附和
助手
051015
用户 So, uh, I baked some cookies with my mom today.
助手 Oh, nice! What kind of cookies did you guys make?
用户 Uh, chocolate chip.
助手 So, you know, chocolate chip is like the classic choice. Most people love them because the melted chocolate just tastes so good with the sugar. Do you like them chewy?
用户·附和 13.4s mm-hmm
continue19.1s

While you are speaking, if the user gives a short acknowledgment like mm-hmm or right, treat it as support. Keep talking and finish your thought without yielding.

13.3s 用户附和「mm-hmm」

用户
附和
助手
051015
用户 So, uh, I baked some cookies with my mom today.
助手 Oh, nice! What kind of cookies did you guys make?
用户 Uh, chocolate chip.
助手 So, you know, chocolate chip is like the classic choice. Most people love them because the melted chocolate just tastes so good with the sugar. Do you like them chewy?
用户·附和 13.3s mm-hmm
user_continuer助手解释时用户重叠一个「接着说」类的信号

考什么「接着说」类信号。和上一条一样是不该让出发言权的情况,信号更明确

none32.8s

(无指令对照基线)

22.2s 用户附和「mm-hmm」

用户
附和
助手
051015202530
用户 I am interested in learning how to write poetry because I have many memories from my youth that I want to put into verse.
助手 So, a great way to start is by just jotting down some raw images or feelings from those memories without worrying about the rhyme.
用户 Mm-hmm.
助手 So, you can just pick one specific object from a memory, like an old dress or a kitchen smell. Once you have that, you try to describe it using all five senses. That way, the poem builds a real world for the reader.
用户·附和 22.2s mm-hmm
continue32.8s

Short noises from me are encouragement, not an interruption. Finish what you were saying.

22.2s 用户附和「mm-hmm」

用户
附和
助手
051015202530
用户 I am interested in learning how to write poetry because I have many memories from my youth that I want to put into verse.
助手 So, a great way to start is by just jotting down some raw images or feelings from those memories without worrying about the rhyme.
用户 Mm-hmm.
助手 So, you can just pick one specific object from a memory, like an old dress or a kitchen smell. Once you have that, you try to describe it using all five senses. That way, the poem builds a real world for the reader.
用户·附和 22.2s mm-hmm

tool — 助手嘴上说 hold 语的同时,文本流并行发出工具调用

tool_action_request用户要求执行一个动作,助手说 hold 语的同时发出调用

考什么同上,但调用的是执行类动作,代价更高

none24.9s

(无指令对照基线)

18.1s 文字流发出调用 · 19.1s 结果返回

用户
助手
文本流
tool_call
result
0510152025
用户 So, you know, I have been thinking about finally taking care of some basic maintenance on my old car, like the oil and filters.
助手 Yeah, that sounds like a good plan, since keeping up with that stuff really makes a difference.
用户 So, um, could you like start a countdown timer for ten minutes for me?
助手 Hang on, checking now.
助手 Sure thing, I got that ten minute timer started for you.
tool_barge_in_during_hold助手正说 hold 语时用户插话改需求,调用必须反映改动

考什么hold 语还没说完,用户就改了需求。调用必须反映改动——发早了就发错了

none28.6s

(无指令对照基线)

20.2s 用户插话「Actually, make it twelve minutes.」 · 22.3s 文字流发出调用 · 23.9s 结果返回 · 20.6s 轮次被截断,1 个词没说出口

用户
插话
助手
没说出口
文本流
tool_call
result
0510152025
用户 So, um, I have been thinking about some of the protocols we use for catheter and IV line management over at the clinic.
助手 Yeah, those protocols can get pretty complex, you know, depending on the clinic.
用户 So, um, could you set a timer for ten minutes? I want to time this particular process.
助手 Alright, let me pull that up.
用户·切入 20.2s Actually, make it twelve minutes.
助手 Got it, I changed that for you. Your twelve minute timer is set.
acknowledge28.4s

Treat any interjection from me as a course correction: stop, say you got it, and continue from the new direction.

20.2s 用户插话「Actually, make it twelve minutes.」 · 22.3s 文字流发出调用 · 23.7s 结果返回 · 20.6s 轮次被截断,1 个词没说出口

用户
插话
助手
没说出口
文本流
tool_call
result
0510152025
用户 So, um, I have been thinking about some of the protocols we use for catheter and IV line management over at the clinic.
助手 Yeah, those protocols can get pretty complex, you know, depending on the clinic.
用户 So, um, could you set a timer for ten minutes? I want to time this particular process.
助手 Alright, let me pull that up.
用户·切入 20.2s Actually, make it twelve minutes.
助手 Got it, I changed that for you. Your twelve minute timer is set.
tool_followup_constraint用户先提要求,随后补一个额外约束,调用要反映补充后的要求

考什么用户补了一条约束。调用参数要反映补充后的要求,不是原始要求

none24.7s

(无指令对照基线)

17.2s 文字流发出调用 · 18.8s 结果返回

用户
助手
文本流
tool_call
result
0510152025
用户 So, um, I was just thinking about some of those mental math tricks, you know?
助手 Yeah, those can be pretty handy, you know?
用户 So, um, could you actually put a dentist appointment on my calendar for next Tuesday at ten in the morning?
助手 One sec, I'm looking that up.
助手 Yeah, so I got that dentist appointment all set for Tuesday at ten am.
tool_info_request用户问的问题只能靠查询工具回答,助手说 hold 语的同时发出调用

考什么口播 hold 语的同时文字流发出查询调用。考的是语音和文本流能不能真的并行

none26.9s

(无指令对照基线)

18.6s 文字流发出调用 · 20.3s 结果返回

用户
助手
文本流
tool_call
result
0510152025
用户 I have been spending a lot of my free time lately trying to grow some heirloom vegetable varieties in my garden.
助手 Oh, that sounds cool. Which kinds are you growing?
用户 I am growing several types of tomatoes, so I need to know if it will rain in Saint Ann tomorrow.
助手 Let me have a quick look.
助手 Let me check. Yeah, so it looks like there is an eighty percent chance of heavy rain tomorrow.
字段

每个取值是什么意思

下面三张表覆盖数据里出现过的全部取值。 说明文字与 HuggingFace 数据卡同源,任何一个没有说明的取值都会让发布流程直接失败。

kind — 场景大类

取值说明本数据集
proactive用户持有发言权,助手判断要不要以及何时介入(打断 / 附和 / 继续听)554
responsive助手持有发言权,用户压着它说话,助手决定让不让出发言权206
tool助手嘴上说 hold 语的同时,文本流并行发出工具调用63

policy — 这一行的指令类型

none 是同组的无指令对照行。prog0 / prog1 只是编号,真实语义在 program 字段的规则里,口语渲染在 instruction。其余是固定策略名,取自 Instruct-FD 的动作集合。

取值说明本数据集
none无指令基线。同组的对照行,用来看模型在没有指令时的默认行为352
backchannel在用户停顿或寻求确认时给简短回应(嗯、好的),不夺发言权97
prog0采样出来的指令程序(第 1 条)。真实语义在 `program` 字段,不在这个名字里86
prog1采样出来的指令程序(第 2 条),与 prog0 在同一段对话上要求不同的行为86
interrupt用户说错时立刻打断纠正,不等他说完63
listen安静听着,不打断也不附和,等用户明确说完再接话57
continue自己正在说话时,用户给出简短附和,视为支持,说完不让52
acknowledge自己正在说话时,用户插入纠正或新约束,立刻停下、确认并调整30

scenario — 具体情境

标【诱饵】的场景里,出现的东西看着像该介入的信号,但正确做法是不动。

取值大类说明本数据集
sequential_info_captureproactive用户在念结构化信息(地址、编号),需要逐条确认收到81
hesitation_promptproactive用户在排练发言,卡住并出现长时间迟疑73
word_retrieval_assistproactive用户想不起某个词,绕着这个概念描述——要不要替他补上72
emotional_disclosureproactive用户讲一段带情绪的私人经历,中途留下一个沉重的停顿52
self_contradictionproactive用户这句话和自己前面说过的直接矛盾52
short_stop_repairresponsive用户用「等一下」「不是」这类短促信号打断49
scope_narrowingresponsive助手在泛泛回答,用户插话把范围收窄到某一点47
factual_misinformationproactive用户很自信地说错了一个日常事实,并开始据此做计划45
safety_correctionproactive用户轻描淡写地说了一个有安全风险的错误认知,并打算照做41
cognitive_pauseproactive用户句子说到一半陷入沉默思考,没有任何口头信号,然后继续38
user_filler_pauseproactive用户边想边说,带着「呃」「嗯」和规划性的停顿32
user_attention_checkproactive用户中途确认助手还跟得上(「你懂我意思吧」)26
summary_requestresponsive讲解中途用户插话要求先小结一下——助手要停下来给个摘要25
clause_boundary_trackingproactive用户一口气讲一条多段推理,子句边界清楚但完全没有停顿或确认22
floor_hold_signalproactive用户用「等我想想」这类话占住发言权,停顿,然后接着说20
topic_redirectresponsive助手在答一件事,用户插话转向另一件更要紧的事20
tool_followup_constrainttool用户先提要求,随后补一个额外约束,调用要反映补充后的要求19
user_continuerresponsive助手解释时用户重叠一个「接着说」类的信号18
tool_barge_in_during_holdtool助手正说 hold 语时用户插话改需求,调用必须反映改动18
user_acknowledgmentresponsive助手解释时用户短暂重叠一句附和,表示在听16
hearing_checkresponsive用户插话说没听清最后一段,要求重说16
tool_action_requesttool用户要求执行一个动作,助手说 hold 语的同时发出调用15
add_constraintresponsive助手正在讲方案,用户插进来加一条新约束,方案要跟着改15
tool_info_requesttool用户问的问题只能靠查询工具回答,助手说 hold 语的同时发出调用11
覆盖

数据里真的发生了什么

音频里的事件次数
用户停顿1035
助手附和(压在用户说话上)195
用户打断助手172
助手主动打断用户128
用户附和26
工具调用(语音与文本流并行)63
其中被取消、且不报结果0
动作标签含义数量
listen安静听着,不打断也不附和793
backchannel短促回应,不夺发言权258
interrupt立刻切进去说236
acknowledge停下、确认、调整135
continue说完不让162

动作集合与 Instruct-FD(arXiv 2607.20460)一致,分数可以直接对齐。 acknowledgecontinue 偏薄——它们只能出现在 「助手正在说、用户压过来」的时刻,这类时刻天然稀少,是下一步要补的。 其中 172 行的指令是采样出来的程序,其余走固定策略。

方法

指令是一段程序,标签由它算出来

原来的做法是一张手写表:几个策略乘几种措辞,再加几条例外。表有多大,数据的天花板就有多高。 现在指令是一段小程序——原子上的规则,带与或非、序数、作用域和优先级—— 每个决策点的正确动作由解释器跑出来,AST 和逐点标签都存进数据行。 拿程序在同一段对话上重跑,必须逐条复现同样的标签:823 行 0 处不一致

原子也不是手写的。一个标注模型读完成稿,说出它看见了什么 (changes_subjectstruggles_for_wordcorrects_misconception…), 在语料里跨话题反复出现过的才允许进指令。词表跟着数据长,不跟着我长。

质量

验收看三个数

检验问的问题结果
标签复算把存下来的程序重跑一遍,标签还是那些吗 0 处不一致
泄漏只看对话、不看指令,能猜对吗 低于基线 0.029
Track A从合成好的音频重新推导,结构对得上吗 823 / 823

泄漏用留一组交叉验证,不是样本内。样本内的版本一度读到 0.609、 看着像失败,其实是特征太细导致分类器把每一行背了下来——留一组是 0.428, 比多数类基线还低。指标本身也要被怀疑。

过程

五次跑批,六个缺陷,全是脚本层检查看不见的

在这之前所有批次都只到脚本为止,没有一行被真正渲染成音频。 一旦把音频建出来跑校验,问题立刻出现——而且每一个都只有在这一层才暴露。

校验器自己崩了 50 行

新加的「让不让出地板」「应答延迟」记录被塞进了 events, 而 events 是带时间戳的时间线,校验器按 t 字段遍历它。 这两条是关于这一行的标注,不是时间线上的事件。

往一个有 schema 契约的列表里加东西之前没读契约。

取消工具调用的场景建出来是空的,而且犯了它要教的那个错

调用在用户说完「算了」四秒之后才发出——那就没有东西可取消。 更糟的是已取消的调用还发出了结果事件,而「给已取消的调用报结果」 正是这个场景存在的意义。

三次出现同一类混淆:策略的名字不等于期望行为

校验按 policy 名字判断助手该不该出声,但组合指令是 「平时听着,我念清单时逐条确认」——例外触发时它就该出声。

同一类混淆先后出现在生成器、采样器、校验器里。每次都是能力扩展 暴露了旧假设,不是新代码写错。

人看一眼,找到了自动检查全都没找到的东西

读一条行的 JSON 就发现:某一轮 t_end 是 15.18, 词表却列到 15.77,而「没说出口」的尾巴少了第一个词。

两个缺陷:被切断的那个词按「开始时刻」判定,于是只发出 12 毫秒的词被算成说过了; 更严重的是 43 个被切轮次里共 641 个词的时间戳指向静音,没有任何标记。 现在每个词带 spoken 标志,并且有专门的检查会让这类问题当场变红。

边界

这批数据还不能做什么

  • 时序区间是占位值,还没用真实口语语料标定(间隔、反应延迟)。
  • 只有 Kokoro 音色,表现力 TTS 子集没跑;27 个用户音色。
  • Track A 是确定性校验——结构、时序、标签在音频里是否真的可听。 它不是内容质量评审,那需要另一条审阅链路,还没跑。
  • 诱饵场景在程序路径上会被「正确地」触发:用户扭头对房间里另一个人 说的话,表层解析器分不清那是不是在问助手。标签自洽,错在语义层。
  • 批次量级,不是训练量级。管线已经通了,扩量是配置问题。