Reading Input and Handling Stream State

Part I · Module 23 · Chapter 23.3 · Reading Input and Handling Stream State

1. The Spin, and What Actually Fixes It

Chapter 23.1 §5 established that a failed extraction consumes nothing. This section is that fact's consequence, measured four ways on the same input — "10 20 oops 30 40" — with a 1,000-iteration cap so the failures report instead of hanging:

// 1. no recovery
while (!in.eof()) { in >> v; use(v); }

// 2. clear() the flag
while (!in.eof()) { in >> v; if (in.fail()) in.clear(); else use(v); }

// 3. clear() the flag AND consume the offending characters
while (!in.eof()) {
    in >> v;
    if (in.fail()) { in.clear(); std::string junk; in >> junk; }
    else use(v);
}

// 4. stop at the first thing you did not understand
while (in >> v) use(v);

  no recovery   : body SPUN (1000 cap), values 2
  clear() only  : body SPUN (1000 cap), values 2
  clear()+skip  : body 5, values 4, tokens skipped 1  <-- terminates
  while(in>>v)  : 2 values, then STOPS at the bad field

Version 2 is the finding. clear() resets the state bits and does not move the read position. The loop clears the failure, attempts the same characters again, fails again, clears again. It is a more confident infinite loop than version 1, and it is what people write when they have learned that clear() is the recovery function. It is half of the recovery.

Version 3 terminates and recovers. The two operations together are the fix: clear() makes the stream usable, and consuming the bad token makes progress. Five iterations, four values, one token skipped — and the four values are the four integers that were actually in the input.

Version 4 is the right answer most of the time. It reads the values up to the first thing it did not understand and stops with the stream in a state you can inspect. If your contract is "this file must be entirely well-formed", version 4 plus a check of what remains is simpler and stricter than any recovery.

The choice between 3 and 4 is a design decision about the input, not a technical one:

Whichever you pick, count what you skipped and report it. Chapter 23.7 §1 measures what happens when you do not: a parser that silently skipped three lines passed a run containing a fatal error.

2. Consuming the Rest of the Line

Skipping one whitespace-delimited token is the right granularity for a stream of numbers. For a line-oriented file the right granularity is the rest of the line, and the idiom is:

in.clear();
in.ignore(std::numeric_limits<std::streamsize>::max(), '\n');

ignore discards characters up to and including the delimiter, or up to the count, whichever comes first. The max() is the conventional way of saying "no limit"; it reads badly and there is no better spelling. For interactive input this is the recovery you want after a failed read, because a person who typed twelve instead of 12 has a whole line of characters you do not want.

A caution: ignore with a delimiter it never finds will consume to the end of input and set eofbit. That is correct behaviour and is occasionally surprising when a file's last line has no newline — §8 covers that case.

3. Mixing >> and getline

These two read whitespace differently, and the result is the most reported stream bug after the eof() loop:

std::istringstream in("42\nthe rest of the line\n");
int n;
in >> n;                      // reads 42, stops AT the newline
std::string line;
std::getline(in, line);       // reads up to the newline -- which is right here

  >> then getline       : n=42  line=""                       len=0  <-- EMPTY
  >> then ws, getline   : n=42  line="the rest of the line"

operator>> skips leading whitespace and stops before the delimiter that ended the value. The newline is still in the stream. getline then reads characters up to the next newline, finds one immediately, and hands you an empty string — having succeeded, so the stream is good and nothing looks wrong.

The fix is std::ws, a manipulator that consumes whitespace:

in >> n >> std::ws;
std::getline(in, line);

This is not the same as in.ignore(..., '\n'). std::ws consumes all leading whitespace including multiple newlines, so it also skips blank lines; ignore consumes exactly to the end of the current line and preserves a following blank line as data. If blank lines are meaningful in your format — a record separator, for instance — std::ws will eat them and ignore will not. Choose knowing which.

The cleanest approach for a line-oriented file is to not mix them at all: read every line with getline, then parse each line with an istringstream. §6 does that, and Chapter 23.5 §5 is the pattern.

4. operator>> Accepts a Prefix

This is the section that changes how you write parsers. Six inputs, each read into an int, with the state captured immediately after the extraction:

  "12"     -> value 12     fail=0 eof=1  leftover ""      
  "12abc"  -> value 12     fail=0 eof=0  leftover "abc"   
  "12 34"  -> value 12     fail=0 eof=0  leftover " 34"   
  "0x1F"   -> value 0      fail=0 eof=0  leftover "x1F"   
  "1e3"    -> value 1      fail=0 eof=0  leftover "e3"    
  " 12"    -> value 12     fail=0 eof=1  leftover ""

All six succeeded. fail=0 in every row. And four of the six did not give the number a caller would have meant:

• "12abc" gives 12 and leaves "abc". A field that is a typo reads as a valid number.
• "0x1F" gives 0 and leaves "x1F". Hexadecimal input into a decimal stream reads the 0 and stops. A register value written as 0x1F silently becomes zero.
• "1e3" gives 1 and leaves "e3". Scientific notation into an int reads the 1.

The "0x1F" row is the dangerous one in this domain, because 0x-prefixed values are how hardware data is written and 0 is a plausible register value. Reading a stimulus file of hex addresses with >> int produces a file of zeros and a clean stream. Chapter 23.8's vector reader uses std::from_chars with base 16 specifically to avoid this.

The root cause is that operator>> is defined to read the longest valid prefix. That is the right definition for a stream — it has to stop somewhere, and the next field may legitimately follow without whitespace — but it means a successful extraction is a statement about the prefix, not about the field. The stream cannot know where you thought the field ended.

So operator>> alone cannot validate. You need one of two things: an explicit check of what is left, or a parser that is told the field's boundaries.

5. from_chars Validates the Whole Field

std::from_chars from <charconv> takes a begin and an end pointer, so it knows where the field stops:

#include <charconv>

bool parseWhole(std::string_view s, int& out)
{
    const char* b = s.data();
    const char* e = s.data() + s.size();
    const auto r = std::from_chars(b, e, out);
    return r.ec == std::errc{} && r.ptr == e;   // no error AND consumed everything
}

  "12"                     -> ACCEPT  value 12     
  "12abc"                  -> REJECT  value 0      (trailing characters)
  ""                       -> REJECT  value 0      (not a number)
  "99999999999999999999"   -> REJECT  value 0      (out of range)
  "-5"                     -> ACCEPT  value -5

One check covers four distinct failures: trailing characters, empty input, out of range, and not-a-number. The two conditions in the return statement do different jobs — r.ec reports whether the conversion failed, and r.ptr == e reports whether it consumed the whole field — and you need both, because "12abc" has r.ec == errc{} and would pass the first test alone.

Three further properties make it the right default for parsing fields:

• It does not allocate and does not throw. Compare std::stoi, which throws std::invalid_argument and std::out_of_range — exceptions for the expected case of a malformed line, which Chapter 22.1 §1 argues against and Chapter 22.7 is the alternative to.
• It is locale-independent. operator>> and std::stod respect the global locale, which means a decimal point can become a comma depending on the environment. For a machine-readable file that is a portability bug waiting for a differently configured machine.
• It takes a base. std::from_chars(b, e, out, 16) parses hexadecimal, which is what the "0x1F" row in §4 needed. You strip the 0x yourself; Chapter 23.8 §1 shows that helper.

The C++17 availability is worth checking in your toolchain for floating point specifically — integer from_chars is widely implemented, and the floating-point overload arrived later in some libraries. Chapter 21.8 §1 is the pattern for probing the toolchain rather than assuming.

6. The Record Loop That Reports What It Did Not Understand

Putting §3, §4 and §5 together on a realistic file — three good lines, one with a bad middle field, a blank, a comment, and one with a field missing:

std::string line;
int lineno = 0, ok = 0, blank = 0, comment = 0, rejected = 0;
std::vector<int> badlines;

while (std::getline(in, line)) {
    ++lineno;
    if (line.empty())   { ++blank;   continue; }
    if (line[0] == '#') { ++comment; continue; }

    std::istringstream ls(line);
    std::string unit;
    long cycles = 0;
    double us = 0;
    if ((ls >> unit >> cycles >> us) && (ls >> std::ws).eof()) ++ok;
    else { ++rejected; badlines.push_back(lineno); }
}

  7 lines: 3 accepted, 1 blank, 1 comment, 2 REJECTED
  rejected line numbers: 3 7

Four decisions in that loop are deliberate and each one prevents a specific defect.

while (std::getline(in, line)) — the read is the condition, per Chapter 23.1 §5. The loop terminates on a bad line rather than spinning, because getline on a line of garbage succeeds; only end of input stops it.

Each line gets its own istringstream. A malformed line cannot leave the main stream in a failed state, so there is no recovery to perform. The failure is contained to one record. This is the structural reason to parse line-by-line rather than field-by-field off the file stream.

Blank lines and comments are classified, not skipped silently. They are counted. A file that is 90% comments because a tool commented out its own output is then visible in the numbers.

(ls >> std::ws).eof() is the whole-line check. Without it, extra fields are accepted:

     "axi0 412 1.25 junk": three fields read = 1, whole line consumed = 0

The three extractions succeeded. junk was simply never looked at. A record format that gains a field in a new tool version would be read by the old parser with the new field ignored — which is sometimes what you want and must always be a decision. The std::ws is there so that trailing spaces and the newline do not count as leftover content.

And the rejected line numbers are retained. rejected: 2 tells a user something is wrong; rejected lines 3 and 7 tells them where. Chapter 22.9 §1's rule about what makes a diagnostic actionable is the same rule.

7. getline's Delimiter, and the Empty Field

std::getline takes an optional delimiter, which makes it a splitter:

std::istringstream in("a,b,,c");
std::string field;
while (std::getline(in, field, ',')) use(field);

  splitting "a,b,,c" on ',': [a] [b] [] [c]   -> 4 fields (the empty one is REAL data)

Four fields, and the third is empty. That is correct: a,b,,c has four comma-separated fields, one of which has no characters. An empty field is data — an unpopulated column, a missing optional value — and a loop that skips empties silently changes the record's shape and misaligns every field after it. If an empty field is invalid in your format, reject it explicitly; do not drop it.

Two limits of this as a CSV reader are worth stating plainly, because it is often used as one. It does not handle quoted fields containing the delimiter, and it does not handle embedded newlines inside quotes. Both are in RFC 4180 and both occur in files exported by real tools. For a format you control, comma-splitting with getline is fine and you should forbid the delimiter in field values. For a file somebody else produced, use a real CSV reader, because a hand-rolled splitter on quoted data fails by misaligning columns, which produces plausible wrong numbers rather than an error.

8. The Last Line Without a Newline

A file whose final line has no terminating newline is common — some editors do it, some tools do it, printf without \n does it — and the question is whether your loop sees that line:

  a final line with no '\n': returned 1, text "no newline at the end"

getline succeeds on an unterminated final line. It reads to end of input, sets eofbit, and returns the stream in a state that converts to true because the read itself worked — which is Chapter 23.1 §3's observation about eof with a successful read, and it is exactly why while (std::getline(in, line)) is correct here while an eof()-controlled loop would drop the last line.

The complementary case is a file ending with a newline. There, getline after the last real line immediately hits end of input and fails, so the loop exits having produced exactly the right number of lines. Both cases work, with no special handling, as long as the read is the loop condition.

9. Reading Interactive Input

std::cin is a stream like any other and everything above applies, with three additions.

A failed read leaves the bad characters in the buffer, so recovery must consume them or the next prompt reads the same garbage. For a person's input the right granularity is the whole line:

int n;
while (!(std::cin >> n)) {
    if (std::cin.eof()) { report("input ended"); break; }
    std::cin.clear();
    std::cin.ignore(std::numeric_limits<std::streamsize>::max(), '\n');
    prompt("that was not a number, try again: ");
}

Note the eof() test inside: without it, a user pressing Ctrl-D gives you an infinite prompt loop, because clear() resets the flag, the stream is still at end of input, and the read fails again. This is §1's lesson in a different costume — clear() without progress does not terminate — and here the progress is impossible, so the loop must detect it and leave.

Prefer getline then parse for anything a human types, because it keeps one input event equal to one line and makes the recovery trivial: you already consumed the line.

std::cin and std::cout are tied by default, so reading flushes the prompt. That is why a prompt written with '\n' rather than std::endl still appears before the read.

10. The Table

11. What to Actually Do

• Make the read the loop condition. while (in >> v) or while (std::getline(in, line)).
• Never clear() without consuming something. Measured: clear() alone spins forever. The pair is clear() plus ignore(...) or >> junk.
• Parse line-by-line through an istringstream. A bad line then cannot poison the file stream, and there is no recovery to write.
• Add (ls >> std::ws).eof() to every record parse. It is the one check that catches a prefix-stop in any field and an unexpected extra field.
• Validate fields with std::from_chars, testing both r.ec and r.ptr == e. It covers bad, empty, partial and out-of-range in one call, does not throw, does not allocate, and ignores the locale.
• Never read 0x-prefixed data with >> int. Measured: it gives 0 and reports success.
• Count and report what you rejected, with line numbers. A silent continue is Chapter 23.7's measured defect.
• Treat an empty field as data, and reject it explicitly if your format forbids it.
• For interactive input, test eof() inside the retry loop, or Ctrl-D becomes an infinite prompt.

12. Mistakes Worth Making Once

in.clear() and nothing else. The most confident infinite loop in C++.

>> then getline. An empty string, and the stream is good.

Trusting fail=0. "12abc" is 12, "0x1F" is 0, "1e3" is 1 — all with a clean stream.

Parsing hex with >> int. Every address becomes 0 and the file looks fine.

Omitting the whole-line check. "axi0 412 1.25 junk" is accepted.

std::stoi in a parse loop. Exceptions for the expected case, and Chapter 22.1 §4 measured what a throw costs when it is frequent.

Skipping empty fields in a split. Every later column shifts.

An eof()-controlled line loop. It drops a final line that has no newline.

Ctrl-D in an interactive retry loop with no eof() check. Infinite prompts.

13. Interview Relevance

The frozen curriculum marks this chapter High, and the question is nearly always "this read loop hangs — why?". The answer has two layers and the second one is what distinguishes a good answer: a failed extraction does not consume the offending characters, so the position never advances; and clear() resets the flags without consuming anything, so adding it makes the loop hang more confidently rather than less. The fix is clear() plus ignore or a token read, and the measurement — 5 iterations and 4 values on an input with one bad token — is a good thing to be able to quote.

A strong follow-up is "how would you validate that a field is an integer?". Saying operator>> cannot do it, because it reads the longest valid prefix and "12abc" succeeds with the value 12, and then reaching for from_chars with both the error code and the end-pointer check, is a complete answer. The "0x1F" case gives it weight in a hardware context.

If record parsing comes up, the (ls >> std::ws).eof() check is the detail to mention — a one-expression conservation check that catches every prefix-stop and every extra field without validating any field individually.

14. What You Should Be Able to Say Now

• That clear() resets state and consumes nothing, so recovery needs both a clear() and a consuming operation — measured, clear() alone spins.
• The three recovery policies — stop, skip and count, reject the file — and when each is right.
• Why >> followed by getline gives an empty line, and that std::ws fixes it while also eating blank lines, which ignore does not.
• That operator>> reads the longest valid prefix and reports success, so six of six test inputs succeeded and four were not the intended value — including "0x1F" reading as 0.
• That std::from_chars needs both r.ec == std::errc{} and r.ptr == e, and that this covers trailing characters, empty, out-of-range and not-a-number.
• That (ls >> std::ws).eof() validates the record even though no field can be validated by the stream, and why that is a conservation check.
• That an empty field between two delimiters is data, and that getline-splitting is not a CSV reader.
• That while (std::getline(...)) reads a final line with no newline.

15. Exercises

Exercise 1 — the four loops. Implement all four versions from §1 with a cap and reproduce the table. Then remove the cap from version 2 and confirm it is the hang, not a slow loop.

Exercise 2 — the prefix table. Reproduce §4 for all six inputs, capturing fail() immediately after the extraction. Then deliberately read the flag after a following getline and observe two rows change — that was a real defect in the first version of this measurement, and it is worth seeing why.

Exercise 3 — validate. Write parseWhole from §5 and run it on the five inputs. Then drop the r.ptr == e condition and find which input starts passing. Add a base-16 overload and parse "0x1F", "1F" and "0xZZ".

Exercise 4 — the record loop. Build §6's loop and feed it a file with a blank line, a comment, a short line, a long line, and a line whose middle field is 12abc. Confirm it rejects exactly two and names their line numbers. Then remove the whole-line check and see which line starts being accepted.

16. Summary

• A failed extraction consumes nothing, so a loop that retries without consuming never terminates. clear() resets the flags and does not consume — measured, it spins on the 1,000-iteration cap exactly like no recovery at all.
• Recovery is clear() plus a consuming read. Measured on one bad token in five: 5 iterations, 4 values, 1 token skipped, and it terminates.
• The alternatives are policies, not techniques: stop at the first bad field for files you own, skip-and-count for a tool's output you must survive, reject the whole file when you are going to act on it.
• >> leaves the newline, so a following getline returns an empty string and the stream stays good. std::ws fixes it; parsing line-by-line avoids it.
• A successful >> does not mean a valid field. All six measured inputs reported fail=0; "12abc" gave 12, "1e3" gave 1, and "0x1F" gave 0 — the last being the one that matters for register data.
• std::from_chars validates the whole field when you check both r.ec and r.ptr == e, covering trailing characters, empty, out-of-range and not-a-number in one call — without throwing, allocating, or consulting the locale.
• (ls >> std::ws).eof() validates the record. Field-level validation is impossible with >>; checking that the fields sum to the whole line is cheap and catches both a prefix-stop and an extra field.
• An empty field is data, and getline-splitting is not an RFC 4180 CSV reader — its failure mode is misaligned columns, which look like plausible wrong numbers.

Chapter 23.4 puts these reads on a real file, where the first thing that can go wrong happens before any of this: the file might not open, and the measured consequence is a report that says zero errors.