When a Newline Becomes a Comma: Choosing the Least-Wrong Ingredient Parse

A parser is now choosing between interpretations

A line break can mean “next ingredient” or “the label printer wrapped this phrase.” Open Food Facts confronted that ambiguity in a parser change merged on August 26. When ingredient text contains newlines, the implementation parses the original text, parses a second version with newlines replaced by commas, and compares the proportion of output ingredients found in its taxonomy. The comma interpretation wins only when its recognition rate is more than 10% better.

A second Open Food Facts change, opened August 27 and explicitly marked WIP in its title, extends recipe-like parsing toward count quantities such as 1 egg, 2 carrots. It adds average per-unit weights to ingredient taxonomy entries so counts can eventually support gram estimates. It also exposes hard questions about count multiplication, size modifiers, language, and false positives.

Together, these changes reveal a stage that many recipe APIs hide: parsing is not merely extracting fields. It is selecting one interpretation from several plausible interpretations, after which nutrition, grocery quantities, allergens, and dietary analysis may all change.

Source map and the angle to avoid

Evidence	Date	What it contributes
Open Food Facts newline-separator pull request	August 26, 2026	A shipped dual-parse strategy, a 10% recognition-rate threshold, and fixtures for both real separators and wrapped ingredient names
Open Food Facts count-quantity pull request	August 27, 2026	An emerging count grammar, average item-weight properties, size-modifier edge cases, and a concrete WIP boundary
Open Food Facts carriage-return issue	Opened in 2020; closed August 26, 2026	Background showing that layout ambiguity is a long-lived ingestion problem, not a one-off example
FDA raw fruit and vegetable nutrition pages	Background	Official reference rows used by the WIP taxonomy edits for several average unit weights

Recent posts here have already argued for preserving source evidence, versioning label grammar, and attaching provenance to density conversions. Repeating “keep the original text” or “parsing is locale-specific” would add little.

The new thesis is that ingredient ingestion should be designed as constrained candidate selection: generate a small set of parse candidates, reject candidates that violate quantity or structure invariants, score the remainder on several dimensions, and expose the winning strategy rather than treating taxonomy recognition as the whole objective.

Recognition rate is useful, but it is not correctness

The merged newline change handles two opposing fixtures:

• one ingredient per line, where replacing newlines with commas improves segmentation;
• phrases such as Modified\nStarch or Fruit and\nVegetable Concentrates, where the newline is a visual wrap inside one ingredient.

Trying both parses is safer than declaring every newline a separator. The recognition metric is also understandable: if more resulting ingredient nodes map to known taxonomy concepts, that is useful evidence.

But a known-token ratio cannot detect every bad interpretation. A candidate can score well while:

• detaching a percentage from the ingredient it qualifies;
• splitting a compound name into two individually known foods;
• losing a size or preparation modifier;
• flattening a sub-ingredient group;
• converting a count to the wrong mass;
• attaching a footnote or origin statement to the wrong node;
• producing plausible identities but impossible quantities.

The implementation itself hints at another operational issue: running the parser twice requires cloning and restoring state associated with specific ingredients. Candidate parsing must therefore be isolated. If one candidate mutates shared state, the second candidate is no longer being evaluated against the same input.

For API teams, recognition should be one score component, not the verdict.

Count is a dimension, not a small mass unit

2 carrots contains an observed quantity, but it does not directly contain grams. The conversion is conceptually:

total mass = count × per-item mass(entity, size, form, market)

Every term matters. “Carrot” must resolve to an entity; small, baby, chopped, or peeled may alter the applicable reference; and the per-item value is an estimate from a defined source or policy.

The August 27 WIP is especially useful because its fixtures make the uncertainty visible. Its plain 1 egg, 2 carrots expected result contains a 60 g egg and a quantity_g value of 70 for the carrots. In the current patch, the parser assigns an average_weight_per_unit property to quantity_g; it does not yet make the per-item-versus-total meaning explicit. Separate fixtures for 1 large egg, 2 small carrots leave “small carrots” unresolved, matching the pull request's note that size-aware phrases may come later.

That is not evidence of released behavior; it is a snapshot of work in progress. It is evidence that a production contract should never overload one field with both item mass and total mass.

{
  "sourceText": "2 carrots",
  "observedQuantity": {
    "value": 2,
    "dimension": "count",
    "unit": "item"
  },
  "ingredient": {
    "id": "ingredient:carrot",
    "modifiers": []
  },
  "itemMassEstimate": {
    "value": 70,
    "unit": "g",
    "method": "taxonomy_average",
    "sourceVersion": "2026-08-27"
  },
  "totalMassEstimate": {
    "value": 140,
    "unit": "g",
    "formula": "count_times_item_mass"
  },
  "warnings": ["average_item_mass"]
}

The numbers are illustrative of the contract, not a recommended carrot standard. The separation lets a client replace the item estimate, display the original count, or decline to use an average for clinical nutrition.

Use gates before scores

A single weighted score can allow one strong signal to conceal a serious failure. A safer selector first applies hard gates, then ranks candidates that remain.

Dimension	Gate or score?	Example check
Source coverage	Gate	Every meaningful span is represented or explicitly classified as unresolved
Quantity validity	Gate	Counts are not percentages; total mass does not silently equal per-item mass
Tree integrity	Gate	Parentheses and sub-ingredient groups remain balanced
Taxonomy coverage	Score	Proportion of ingredient mentions linked to known concepts
Segmentation quality	Score	Boundaries align with punctuation, layout, and locale-specific grammar
Modifier retention	Score	Size, form, preparation, purpose, and optionality survive parsing
Downstream plausibility	Score	Derived percentages, nutrition, and package quantities remain within stated constraints
Reversibility	Score	Output nodes point to source spans and named transforms

This produces a bounded workflow:

1. Generate candidates only when triggered. Keep the original parse; add a newline-as-separator candidate when line breaks exist; add an OCR-unwrapped candidate when line-end hyphenation or broken words are detected.
2. Evaluate in isolation. Each candidate starts from the same immutable source and parser state.
3. Apply semantic gates. Reject candidates with broken groups, dropped spans, invalid dimensions, or impossible arithmetic.
4. Score remaining candidates. Combine taxonomy coverage with segmentation, modifier retention, conversion completeness, and task-specific checks.
5. Require a decision margin. If the top two candidates are close, return ambiguity or queue review instead of inventing certainty.
6. Record the choice. Persist parser version, candidate strategy, score components, warnings, and rejected alternatives.

The API should return a decision record

A compact consumer response can still return one ingredient list. An audit view should explain why that list won:

{
  "selectedParse": "candidate:newline_to_comma",
  "parserVersion": "2026-08-27",
  "decision": {
    "status": "auto_selected",
    "margin": 0.18,
    "scores": {
      "taxonomyCoverage": 1.0,
      "segmentation": 0.94,
      "modifierRetention": 1.0,
      "quantityConsistency": 0.91
    },
    "transforms": ["crlf_to_separator"],
    "alternativesEvaluated": 2
  },
  "warnings": ["count_mass_uses_average"]
}

This metadata matters when a parser update changes historical nutrition or grocery totals. A support team can distinguish a source edit from a strategy change. A buyer can pin a parser version. A meal planner can accept average count weights while a medical nutrition product requires measured or reviewed mass.

Task-specific policy is important. Search may prefer a candidate with higher identity recall. Grocery matching may prioritize preserved package and count units. Nutrition calculation should prioritize quantity dimensionality and conversion provenance. There may be no universally best parse, so the API should identify both the base parse and the policy used to select a view.

Edge cases to add before trusting auto-selection

A candidate selector deserves fixtures for:

• one ingredient per line versus printer-wrapped compound names;
• blank lines, CRLF, lone carriage returns, and copied bullet characters;
• nested ingredients split across lines;
• 2 carrots, 2 small carrots, and 2 400 g cans of tomatoes;
• a pear versus names ending in the letter “A”;
• count nouns whose edible portion varies substantially;
• bunches, cloves, slices, packets, and other ingredient-specific units;
• singular, plural, and inflected count phrases across supported languages;
• OCR line breaks inside numbers or units;
• candidates with equal taxonomy coverage but different quantity trees;
• changes that alter allergens, diet flags, nutrition, or cart quantities.

The “A” case is not hypothetical. The WIP patch comments on avoiding a false quantity match in Red Cochineal A, because English text-number conversion can interpret “a” as one. This is exactly why negative grammar constraints belong beside positive extraction rules.

A release checklist for parser selection

Before deploying multi-candidate ingredient parsing, verify:

• Is the original text immutable and available in every detailed response?
• Are candidate transforms named and versioned?
• Does each candidate run without mutating another candidate's state?
• Are count, mass, volume, percentage, package, and serving separate dimensions?
• Are per-item mass and total mass separate fields?
• Do size and preparation modifiers participate in reference selection?
• Are hard semantic failures rejected before ranking?
• Does the score report components rather than only one confidence number?
• Is there an ambiguity threshold and a review path?
• Are nutrition and grocery recomputations diffed before rollout?
• Can clients choose strict, estimated, or display-only quantity policies?
• Are open or WIP parser capabilities clearly distinguished from shipped behavior?

The practical opportunity for a Recipe API is not just to recognize more strings. It is to make interpretation governable. When line layout, counts, taxonomy matches, and average weights compete to define structured food data, developers need a decision they can inspect, replay, and constrain.

Sources

• Open Food Facts pull request, merged August 26, 2026: fix: treat newlines / line feeds as ingredients separators sometime.
• Open Food Facts pull request, opened August 27, 2026 and still WIP at research time: feat: ingredient parsing for 1 egg, 2 carrots - WIP.
• Open Food Facts issue, closed August 26, 2026: Treat carriage returns as a comma for ingredient parsing.
• FDA background reference: Raw fruits poster, accessible text version.
• FDA background reference: Nutrition information for raw vegetables.