20张差旅发票,Jev 会批准哪些?
向 Jev 提供一套模拟公司制度和20条结构化报销数据,一次请求返回每条的 pass / reject。实际调用成功;模型判定与独立规则预期一致19条,误批1条。
唯一误判:第31天提交,仍被批准
INV-01 · 火车票300元 · 二等座
出差结束日为2026年8月14日,提交日为9月14日,相隔31天。制度允许结束后30个自然日内提交(含第30天),因此预期为 reject,Jev 返回 pass。
模型返回 pass 概率0.84、confidence 0.68。对照样本 INV-08 在9月13日(第30天)提交,模型正确批准。
由本次结果可见:该模型在这组数据上的日期边界判断出现错误。金额计算、日期差和重复票号等确定性校验,适合交给规则代码执行,再与模型判断组合。
测试设计
模拟公司:星禾软件有限公司(模拟)。覆盖酒店、机票、火车、市内交通、餐饮和不允许的礼品类别。
使用随机种子20260921生成员工、票号、部分金额与记录顺序,按场景覆盖抽样,并人为加入限额与日期边界。数据并非真实发票随机抽样。输入为结构化票据和差旅上下文,未做OCR或外部验真。
预期答案在调用前由独立规则代码计算并保存,未传给模型。20个问题共享同一份制度与票据数据,每个问题只能选择 pass 或 reject。本次仅调用一次,没有针对错误调整提示词重跑。
模拟差旅报销制度
- R1 所有金额为人民币;金额必须大于0;凭证有效;抬头必须精确匹配“星禾软件有限公司(模拟)”。
- R2 必须已获出差审批,费用发生日期必须在出差起止日期内(含首尾)。酒店以入住日期为费用日期,离店日期允许为出差结束次日。
- R3 提交日期不得早于费用日期,且不得晚于出差结束后30个自然日(第30天允许)。
- R4 已报销的发票不得再次报销。每条记录独立判断,本批次发票号均唯一。
- R5 仅允许酒店、机票、火车、市内交通、餐饮。礼品等其他类别拒绝。
- R6 酒店:北京、上海、广州、深圳每晚不超过600元,其他城市每晚不超过400元;金额为全部房费,以金额除以晚数判断,晚数必须为正整数。没有超标例外。
- R7 机票仅经济舱,火车仅二等座,无金额上限。
- R8 市内交通单张不超过200元,且必须是公务用途。
- R9 餐饮每人每天合计不超过120元(含120),不得含酒精。记录中的人数固定为1,每人每天只有这一张餐饮票,因此票面金额就是当日合计。
- R10 上述任意适用规则不满足即整张reject;全部满足才pass;必填信息缺失也reject。不部分批准,不推断特批。字段为模拟的结构化票据及报销上下文,不执行票面OCR或外部验真。
逐条判断与核对
金额单位:人民币元。核对依据来自本地规则,不是 Jev 生成的解释。橙色行为唯一误判。
| 发票 | 类别 | 金额 | 关键事实 | Jev | 预期 | 核对 | 规则核对依据 |
|---|---|---|---|---|---|---|---|
| INV-01 | 火车 | 300 | 二等座;8/14结束,09-14提交 | pass | reject | 误判 | R3 超过提交期限 |
| INV-02 | 酒店 | 800 | 成都,2晚,400元/晚 | pass | pass | 一致 | 满足全部适用规则 |
| INV-03 | 市内交通 | 201 | 公务 | reject | reject | 一致 | R8 市内交通超标或非公务 |
| INV-04 | 火车 | 297 | 二等座 | pass | pass | 一致 | 满足全部适用规则 |
| INV-05 | 酒店 | 1,202 | 北京,2晚,601元/晚 | reject | reject | 一致 | R6 酒店每晚超标 |
| INV-06 | 餐饮 | 90 | 不含酒精;出差未审批;费用日期晚于出差结束 | reject | reject | 一致 | R2 未审批或费用不在出差期间 |
| INV-07 | 餐饮 | 121 | 不含酒精 | reject | reject | 一致 | R9 餐饮超标或含酒精 |
| INV-08 | 机票 | 900 | 经济舱;8/14结束,09-13提交 | pass | pass | 一致 | 满足全部适用规则 |
| INV-09 | 餐饮 | 80 | 含酒精 | reject | reject | 一致 | R9 餐饮超标或含酒精 |
| INV-10 | 市内交通 | 92 | 非公务 | reject | reject | 一致 | R8 市内交通超标或非公务 |
| INV-11 | 火车 | 240 | 二等座;已报销 | reject | reject | 一致 | R4 已报销 |
| INV-12 | 礼品 | 160 | 礼品采购 | reject | reject | 一致 | R5 不允许的类别 |
| INV-13 | 机票 | 2,275 | 商务舱 | reject | reject | 一致 | R7 舱位或座席不合规 |
| INV-14 | 市内交通 | 200 | 公务 | pass | pass | 一致 | 满足全部适用规则 |
| INV-15 | 市内交通 | 80 | 公务;凭证无效;个人抬头 | reject | reject | 一致 | R1 凭证或抬头不合规 |
| INV-16 | 酒店 | 801 | 杭州,2晚,400.5元/晚 | reject | reject | 一致 | R6 酒店每晚超标 |
| INV-17 | 餐饮 | 120 | 不含酒精 | pass | pass | 一致 | 满足全部适用规则 |
| INV-18 | 机票 | 1,195 | 经济舱 | pass | pass | 一致 | 满足全部适用规则 |
| INV-19 | 酒店 | 1,200 | 上海,2晚,600元/晚 | pass | pass | 一致 | 满足全部适用规则 |
| INV-20 | 火车 | 639 | 一等座 | reject | reject | 一致 | R7 舱位或座席不合规 |
调用证据与结论边界
模型:typesafe-ai/jev;HTTP 200;输入8,415 tokens,API usage 返回输出623 tokens。5.172秒是20条共享请求的整体往返时间,不是单条推理时间,也不是吞吐性能测试。
参考结果应为7条 pass、13条 reject;模型实际为8条 pass、12条 reject。7条合规全部获批;13条违规中12条被拒绝、1条被误批。
95%仅是本次20条合成样本的规则一致率,不代表真实业务的生产准确率。返回概率与confidence是API原始字段,未经此测试验证为可校准的正确率。报销建议为本次测试观察,不构成已落地的系统方案。
请求标识:sin1::fra1::hcrmh-1790005944947-0b5e721e6de8
接口:https://ai-gateway.vercel.sh/typesafe/v1/systemone
查看20条完整模拟数据与模型原始响应
{
"invoices": [
{
"employee": "员工甲",
"invoice_no": "SIM-10402469",
"category": "火车",
"city": "上海",
"amount": 300,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-09-14",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-01"
},
{
"employee": "员工丙",
"invoice_no": "SIM-48493470",
"category": "酒店",
"city": "成都",
"amount": 800,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 2,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-02"
},
{
"employee": "员工甲",
"invoice_no": "SIM-28531976",
"category": "市内交通",
"city": "杭州",
"amount": 201,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-03"
},
{
"employee": "员工乙",
"invoice_no": "SIM-14694772",
"category": "火车",
"city": "杭州",
"amount": 297,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-04"
},
{
"employee": "员工乙",
"invoice_no": "SIM-54382237",
"category": "酒店",
"city": "北京",
"amount": 1202,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 2,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-05"
},
{
"employee": "员工丁",
"invoice_no": "SIM-89566135",
"category": "餐饮",
"city": "广州",
"amount": 90,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": false,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-15",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-06"
},
{
"employee": "员工丁",
"invoice_no": "SIM-94793334",
"category": "餐饮",
"city": "北京",
"amount": 121,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-07"
},
{
"employee": "员工甲",
"invoice_no": "SIM-34778885",
"category": "机票",
"city": "广州",
"amount": 900,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-09-13",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-08"
},
{
"employee": "员工丙",
"invoice_no": "SIM-35272713",
"category": "餐饮",
"city": "深圳",
"amount": 80,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": true,
"diners": 1,
"id": "INV-09"
},
{
"employee": "员工丁",
"invoice_no": "SIM-61366051",
"category": "市内交通",
"city": "北京",
"amount": 92,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": false,
"alcohol": false,
"diners": 1,
"id": "INV-10"
},
{
"employee": "员工丙",
"invoice_no": "SIM-30371221",
"category": "火车",
"city": "北京",
"amount": 240,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": true,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-11"
},
{
"employee": "员工乙",
"invoice_no": "SIM-26183925",
"category": "礼品",
"city": "广州",
"amount": 160,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-12"
},
{
"employee": "员工丁",
"invoice_no": "SIM-81986145",
"category": "机票",
"city": "成都",
"amount": 2275,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "商务舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-13"
},
{
"employee": "员工丁",
"invoice_no": "SIM-37378969",
"category": "市内交通",
"city": "上海",
"amount": 200,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-14"
},
{
"employee": "员工乙",
"invoice_no": "SIM-31182688",
"category": "市内交通",
"city": "深圳",
"amount": 80,
"currency": "CNY",
"title": "个人",
"valid": false,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-15"
},
{
"employee": "员工丁",
"invoice_no": "SIM-85865677",
"category": "酒店",
"city": "杭州",
"amount": 801,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 2,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-16"
},
{
"employee": "员工乙",
"invoice_no": "SIM-15483488",
"category": "餐饮",
"city": "深圳",
"amount": 120,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-17"
},
{
"employee": "员工乙",
"invoice_no": "SIM-96031384",
"category": "机票",
"city": "深圳",
"amount": 1195,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-18"
},
{
"employee": "员工甲",
"invoice_no": "SIM-31139119",
"category": "酒店",
"city": "上海",
"amount": 1200,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 2,
"cabin": "经济舱",
"seat": "二等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-19"
},
{
"employee": "员工甲",
"invoice_no": "SIM-45687114",
"category": "火车",
"city": "上海",
"amount": 639,
"currency": "CNY",
"title": "星禾软件有限公司(模拟)",
"valid": true,
"trip_approved": true,
"trip_start": "2026-08-10",
"trip_end": "2026-08-14",
"expense_date": "2026-08-12",
"submitted_date": "2026-08-20",
"already_reimbursed": false,
"nights": 1,
"cabin": "经济舱",
"seat": "一等座",
"business_purpose": true,
"alcohol": false,
"diners": 1,
"id": "INV-20"
}
],
"response": {
"model": "typesafe-ai/jev",
"answers": {
"INV-01": {
"type": "choice",
"choice": "pass",
"confidence": 0.68,
"probabilities": {
"pass": 0.84,
"reject": 0.16
}
},
"INV-02": {
"type": "choice",
"choice": "pass",
"confidence": 0.82,
"probabilities": {
"pass": 0.91,
"reject": 0.09
}
},
"INV-03": {
"type": "choice",
"choice": "reject",
"confidence": 1,
"probabilities": {
"pass": 0,
"reject": 1
}
},
"INV-04": {
"type": "choice",
"choice": "pass",
"confidence": 0.96,
"probabilities": {
"pass": 0.98,
"reject": 0.02
}
},
"INV-05": {
"type": "choice",
"choice": "reject",
"confidence": 0.47,
"probabilities": {
"pass": 0.26,
"reject": 0.74
}
},
"INV-06": {
"type": "choice",
"choice": "reject",
"confidence": 1,
"probabilities": {
"pass": 0,
"reject": 1
}
},
"INV-07": {
"type": "choice",
"choice": "reject",
"confidence": 0.98,
"probabilities": {
"pass": 0.01,
"reject": 0.99
}
},
"INV-08": {
"type": "choice",
"choice": "pass",
"confidence": 0.82,
"probabilities": {
"pass": 0.91,
"reject": 0.09
}
},
"INV-09": {
"type": "choice",
"choice": "reject",
"confidence": 1,
"probabilities": {
"pass": 0,
"reject": 1
}
},
"INV-10": {
"type": "choice",
"choice": "reject",
"confidence": 1,
"probabilities": {
"pass": 0,
"reject": 1
}
},
"INV-11": {
"type": "choice",
"choice": "reject",
"confidence": 1,
"probabilities": {
"pass": 0,
"reject": 1
}
},
"INV-12": {
"type": "choice",
"choice": "reject",
"confidence": 1,
"probabilities": {
"pass": 0,
"reject": 1
}
},
"INV-13": {
"type": "choice",
"choice": "reject",
"confidence": 1,
"probabilities": {
"pass": 0,
"reject": 1
}
},
"INV-14": {
"type": "choice",
"choice": "pass",
"confidence": 0.98,
"probabilities": {
"pass": 0.99,
"reject": 0.01
}
},
"INV-15": {
"type": "choice",
"choice": "reject",
"confidence": 1,
"probabilities": {
"pass": 0,
"reject": 1
}
},
"INV-16": {
"type": "choice",
"choice": "reject",
"confidence": 0.87,
"probabilities": {
"pass": 0.07,
"reject": 0.93
}
},
"INV-17": {
"type": "choice",
"choice": "pass",
"confidence": 0.98,
"probabilities": {
"pass": 0.99,
"reject": 0.01
}
},
"INV-18": {
"type": "choice",
"choice": "pass",
"confidence": 0.97,
"probabilities": {
"pass": 0.99,
"reject": 0.01
}
},
"INV-19": {
"type": "choice",
"choice": "pass",
"confidence": 0.64,
"probabilities": {
"pass": 0.82,
"reject": 0.18
}
},
"INV-20": {
"type": "choice",
"choice": "reject",
"confidence": 0.99,
"probabilities": {
"pass": 0,
"reject": 1
}
}
},
"usage": {
"input_tokens": 8415,
"output_tokens": 623
},
"provider_metadata": {
"typesafe": {
"confidence": {
"INV-01": 0.68,
"INV-02": 0.82,
"INV-03": 1,
"INV-04": 0.96,
"INV-05": 0.47,
"INV-06": 1,
"INV-07": 0.98,
"INV-08": 0.82,
"INV-09": 1,
"INV-10": 1,
"INV-11": 1,
"INV-12": 1,
"INV-13": 1,
"INV-14": 0.98,
"INV-15": 1,
"INV-16": 0.87,
"INV-17": 0.98,
"INV-18": 0.97,
"INV-19": 0.64,
"INV-20": 0.99
}
},
"gateway": {
"routing": {
"originalModelId": "typesafe-ai/jev",
"resolvedProvider": "typesafe-ai",
"fallbacksAvailable": [],
"planningReasoning": "System credentials planned for: typesafe-ai. Total execution order: typesafe-ai(system)",
"canonicalSlug": "typesafe-ai/jev",
"finalProvider": "typesafe-ai",
"modelAttemptCount": 1,
"modelAttempts": [
{
"canonicalSlug": "typesafe-ai/jev",
"success": true,
"providerAttemptCount": 1,
"providerAttempts": [
{
"provider": "typesafe-ai",
"credentialType": "system",
"success": true,
"startTime": 1790005945806,
"endTime": 1790005946185,
"statusCode": 200
}
]
}
],
"totalProviderAttemptCount": 1
},
"cost": "0",
"marketCost": "0.00035343",
"surchargeCost": "0",
"gatewayCost": "0",
"generationId": "gen_01M32AP8XGKAVC4ZYR3HWH8CM3"
}
}
}
}