Question Plans

Why Questions, Order, and Measurement Source Versions Must Stay Fixed

Questions, order, sources, models, and versions must remain explicit before results can qualify for comparison.

GrayRhino AI Methodology and Research2 min read

Publisher: GrayRhino AI · Owner: Methodology and content owner · Review: Fact and methodology reviewer · Reviewed 2026-09-22

Comparable change starts with comparable conditions

External fact

Baidu's standard Intelligent Search Generation API includes search configuration, while web_summary is a separate endpoint. Interfaces and configurations from one vendor are not automatically the same measurement condition.

GrayRhino AI method

Question text, order, input facts, source plan, and analysis rules jointly determine comparison eligibility. A change creates a new version instead of silently rewriting history.

A version is more than a display number

GrayRhino AI method

Question fingerprints are recalculated from normalized content; source plans preserve the actual model and interface. Results from different versions are not forced into one trend. Changing a single confirmed audience fact may alter question text, its reason, and its fingerprint. Keeping the old version number would make historical evidence appear more comparable than it is.

When a team asks why this month's result changed, line up question wording, order, market, source configuration, and analysis rules first. Preserve both records if a condition differs, but label the direct rise-or-fall calculation ineligible.

A small input change can create a new question

Synthetic example

Synthetic teaching example: one question asks how small-batch manufacturers choose equipment monitoring; the next asks how retail stores choose it. The grammar is similar, but the audience is different, so the brand candidates in the answers are not a like-for-like trend.

The same issue arises if an otherwise unchanged Baidu standard request switches its search configuration from baidu_search_v1 to baidu_search_v2. Record the configuration change rather than presenting both requests as one condition. This is a comparability example, not a claim that GrayRhino AI currently runs both configurations.

How GrayRhino AI measures this issue

GrayRhino AI method

Question cards use controlled confirmed facts rather than free-text rewriting. When a first measurement starts, every question binds to every planned measurement source as an expected task. Two sources answering one comparison question create two response tasks, but still only one question in a question-level competitor metric.

Fixed conditions do not guarantee identical wording

Limitation

Generated answers can still vary. Fixed conditions make differences interpretable; they do not promise verbatim reproduction.

Sources

Related reading