The Stream Model

Part I · Module 23 · Chapter 23.1 · The Stream Model

1. One Abstraction, Four Destinations

A stream is a sequence of characters with a current position and a record of how things are going. That is the whole model. The standard library gives it to you through a small hierarchy:

std::ios_base          // the state bits and the formatting flags
  std::ios             // + a character type
    std::istream       // + operator>>, get, getline, read, seekg
      std::ifstream    //   characters come from a file
      std::istringstream // characters come from a std::string
    std::ostream       // + operator<<, put, write, seekp, flush
      std::ofstream    //   characters go to a file
      std::ostringstream // characters go to a std::string

The important consequence is that a function taking std::istream& works on a file, on a string, and on std::cin without knowing which it has:

int countRecords(std::istream& in)
{
    std::string line;
    int n = 0;
    while (std::getline(in, line))
        if (!line.empty()) ++n;
    return n;
}

That function is testable without a file. You hand it an std::istringstream holding three lines, you assert it returns 3, and the test runs in microseconds with nothing on disk. The same function in production receives an std::ifstream. Chapter 23.5 is built entirely on that substitution, and Chapter 23.8 uses it to make a model testable in a regression flow. Write your stream-consuming functions against std::istream&, not against std::ifstream&, and you get that for free.

Here is the same operator reaching three different destinations:

  ostringstream : to a string
  cout          : (the next line)
  to the console, through the same operator<<

2. The Four Bits

The state is four flags, and the library exposes them through five functions. Reading them carefully once saves a great deal of confusion later:

Two of those entries do real work. fail() is true when badbit is set, not only when failbit is — so fail() is the broad "something went wrong" question and bad() is the narrow "the stream is unusable" one. And operator bool is defined as !fail(), which means it ignores eofbit entirely. A stream that has reached the end of its input but whose last read succeeded converts to true.

Measured, that distinction is not academic:

  failbit only (bad input):   fail=1 bad=0
  badbit set (stream broken): fail=1 bad=1  -> fail() is TRUE when bad is
  fail() means failbit OR badbit; bad() means only badbit

The practical rule that falls out: check the stream, not a specific bit. if (!in) is almost always what you want, because it asks the question you actually care about — did that work — and it keeps working when the reason changes from bad data to a broken file.

3. The Bits Through One Extraction's Life

Abstract flag descriptions do not stick. Here is a single std::istringstream holding "42 abc", with all four bits printed after every operation:

std::istringstream in("42 abc");
int n = -1;
in >> n;          // reads 42
int m = -1;
in >> m;          // "abc" is not an int
in.clear();       // reset the flags
std::string word;
in >> word;       // reads "abc"
std::string more;
in >> more;       // nothing left

  fresh stream                       good=1 eof=0 fail=0 bad=0   as bool: true
    read an int -> 42
  after reading 42                   good=1 eof=0 fail=0 bad=0   as bool: true
    tried to read an int -> 0  (was -1 before the read)
  after FAILING on "abc"             good=0 eof=0 fail=1 bad=0   as bool: FALSE
  after clear()                      good=1 eof=0 fail=0 bad=0   as bool: true
    read it as a string -> abc
  after reading "abc"                good=0 eof=1 fail=0 bad=0   as bool: true
  after reading past the end         good=0 eof=1 fail=1 bad=0   as bool: FALSE

Four things in that transcript are worth naming.

Failing on "abc" set failbit and left eofbit clear. The characters are still there. Nothing was consumed. This is the single most important fact in the chapter and §5 is about its consequence.

clear() reset the flags and did not touch the characters. It is a flag operation, not an input operation. Clearing a failure without consuming the offending characters leaves you exactly where you were.

After successfully reading "abc" the stream has eof=1 fail=0 and converts to true. The read worked; it also happened to reach the end while scanning for the end of the word. A loop that stops on eof() would stop here, after a successful read, having already used the value — which is fine — but the same logic on a stream with a trailing newline behaves differently, and §5 measures that.

Reading past the end set both eofbit and failbit. This is the normal, expected end of input. It is not an error condition to report; it is how input ends.

4. What a Failed Read Leaves in the Variable

When an extraction fails, what happens to the variable? C++11 changed the answer, and the modern one is the safer of the two:

  value after a failed read: 0   (C++11+ writes 0, pre-11 left it alone)
  unsigned from "-1": 4294967295  fail=0  (not an error -- it wraps)
  short from "99999": 32767  fail=1  (OUT OF RANGE -> fail + max)

Since C++11 a failed extraction writes zero into the target. Before C++11 it left the variable untouched, which meant a failed read of an uninitialised variable gave you garbage and no indication. The new rule is better but it has a sharp edge: a value of 0 after a read tells you nothing by itself, because it is both a legitimate input and the failure marker. You must check the stream.

The other two rows are the ones that surprise people, and they are about ranges rather than syntax.

Reading -1 into an unsigned succeeds and gives 4294967295. The stream applies the ordinary unsigned conversion. There is no failure bit, because nothing failed by the library's definition — it read a valid integer and converted it. If your input file can contain a negative number where your struct has an unsigned, the stream will not tell you.

Reading 99999 into a short sets failbit and writes the maximum. Out-of-range is a failure, and the value written is the clamped boundary rather than zero. So the two range problems behave oppositely: signed-to-unsigned wraps silently, too-large-for-the-type fails loudly. Neither is something you can see by looking at the resulting number alone.

Chapter 23.3 §4 shows the tool that removes all of this ambiguity in one call, and Chapter 9.9 §4 measured the related silent conversion for plain enumerations.

5. eof Is Not Failure, and the Loop That Never Ends

Three inputs, two loop conditions, and a result that decides how you write every read loop for the rest of your career:

// A: the correct loop
int v;
while (in >> v) use(v);

// B: the loop that looks equivalent
while (!in.eof()) { in >> v; use(v); }

  "1 2 3"   (ends on the digit)  while(in>>v): 3 |  while(!eof): body 3, valid 3
  "1 2 3\n" (ends on a newline)  while(in>>v): 3 |  while(!eof): body 4, valid 3  <-- one extra
  "1 2 x"   (bad third field)   while(in>>v): 2 |  while(!eof): body SPUN (hit the 1000 cap), valid 2  <-- INFINITE LOOP

Read the three rows in order, because they are three different failures of the same mistake.

On "1 2 3" the two loops agree. Reading the final 3 consumes the last character and sets eofbit as a side effect of scanning, so loop B exits at the right moment. This is why the bug survives code review: on the input someone tried, it works.

On "1 2 3\n" loop B runs its body four times for three values. The trailing newline means the read of 3 stopped before the end, so eofbit was not set, so the loop ran again, and that fourth read failed. Loop B then used v — which, per §4, the failed read set to 0. A fourth record appeared in the output with every field zero. Almost every text file ends with a newline, so this is the common case, not the corner case.

On "1 2 x" loop B never terminates. The read of x fails without consuming anything and without reaching the end of input, so eof() stays false and the stream position never moves. The loop spins on the same character forever. I hit this for real while writing the measurement program: the first version of this experiment had no iteration cap and the build simply stopped responding. That is what this bug looks like in production — not a wrong number, a hang.

6. The Buffer Is Part of the Model

A stream is characters plus state, and between your operator<< and the file there is a buffer. That buffer is observable, and not knowing about it produces two distinct bugs.

Watching a file's size from outside the stream while writing to it:

  after 1000 chars        : 0 bytes on disk
  after 2000 chars        : 0 bytes on disk
  after an explicit flush : 2000 bytes on disk
  after 1000 more         : 2000 bytes on disk
  after close()           : 3000 bytes on disk

Two thousand characters had been written and the file was empty. "Written" and "saved" are different events. A program that writes a diagnostic and then crashes may leave nothing behind, and the diagnostic you most want is the one from the run that crashed. Chapter 23.4 §5 measures the same effect through std::filesystem::file_size, and Chapter 22.9's logging advice to flush on the error path is exactly this.

The block size is measurable too:

  the first bytes reached the disk after about 4096 chars,
  in a block of 4096 bytes
  -> that block size is the amount a crash can lose

Four kilobytes on this libc++. That number is the upper bound on how much output a crash can discard.

7. endl Against a Newline, Measured

std::endl writes a newline and flushes. '\n' writes a newline. The bytes in the file are identical; the cost is not:

The files are byte-identical — same size, same contents. Each number is already a minimum of nine repetitions within a run, and the table reports the median across nine whole-process runs with the range beside it. The range is wide because a flush is a system call, and system calls are the part of this machine most sensitive to whatever else is running; the ratio never came near 1, which is the claim that matters. I give the range rather than a single run because a single run of this experiment reported 11.9×, which is below everything the nine runs produced.

So std::endl costs more than an order of magnitude per line and produces the same file. Use '\n'. Flush deliberately — at the end of a phase, on the error path, before a long computation during which you want the log on disk — and not once per line out of habit. The exception is interactive output a human is watching, where you want the flush; std::cout is also line-buffered to a terminal by default, which is why the habit is harmless there and expensive everywhere else.

8. The File Is Not the Cost

It is natural to assume writing to a file is slow and writing to memory is fast. Measured on the same twenty thousand formatted lines:

The file costs about a quarter more than the same formatting into memory. The formatting — parsing the format, converting the integer, padding the field — is most of the work, and the destination is a minority of it. Chapter 23.5 §2 decomposes that formatting cost directly and finds the same thing from the other side.

This matters for where you look when output is slow. The instinct is to batch the writes. The measurement says the writes were never the problem; the per-line formatting is, and §7's flush is, and Chapter 22.9 §2's decide-before-formatting is.

9. Reading and Writing the Same Stream

std::stringstream and std::fstream do both directions, and the position is shared in ways worth knowing before you rely on them. For files, Chapter 23.4 §6 measures std::ios::in | std::ios::out. For strings the common use is simpler: build in an ostringstream, then parse the result in an istringstream, as separate objects.

std::ostringstream out;
out << "axi0 412 1.2345";

std::istringstream in(out.str());
std::string unit;
long cycles;
double us;
in >> unit >> cycles >> us;

That is the honest pattern and it is what Chapter 23.5 builds on. Mixing reads and writes on one stringstream without repositioning works, but the rules about when the write position affects the read position are the kind of detail that makes code hard to review. Two objects and an explicit str() cost one copy and remove the question.

10. What Sets badbit

failbit comes from data the stream could not interpret. badbit comes from the stream itself failing, and it is rarer. In practice you will see it from a stream whose underlying buffer is gone or was never there — an ifstream on a file that could not be opened, a stream whose rdbuf() is null — and from the library when an exception escapes the buffer.

The reason to care is that badbit is unrecoverable in a way failbit is not. You can clear() a failbit, consume the bad characters, and carry on; Chapter 23.3 §1 measures exactly that recovery. There is nothing useful to do after badbit except report and stop, because the characters are not coming.

You can also ask the stream to throw instead of setting flags:

std::ifstream in("report.txt");
in.exceptions(std::ios::badbit | std::ios::failbit);

This is worth knowing about and is rarely the right choice for parsing, because expected failures — a line that does not match, the end of input — then arrive as exceptions, and Chapter 22.1 §1 is about why that is the wrong mechanism for a frequent, expected outcome. Enabling it for badbit alone is more defensible: that is genuinely exceptional. Enabling it for failbit turns every end-of-input into a throw.

11. The Table

12. What to Actually Do

• Take std::istream& and std::ostream& in your functions. You get file, string and console for one signature, and your tests need no files.
• Write while (in >> x) or while (std::getline(in, line)). Never while (!in.eof()). Measured: it fabricates a record on a trailing newline and hangs on a bad field.
• Check the stream, not a bit — if (!in) — unless you specifically need to distinguish end-of-input from bad data, in which case test eof() after the loop.
• Treat 0 after a read as meaningless until you have checked the stream. A failed extraction writes it.
• Use '\n', and flush on purpose. std::endl is a newline plus a flush, measured at about 17× for identical bytes.
• Flush before anything that might not return, and on the error path. Up to 4 KB of diagnostics is otherwise lost exactly when you need it.
• Reach for std::from_chars when a field must be validated, which Chapter 23.3 §4 measures — >> accepts prefixes.

13. Mistakes Worth Making Once

while (!in.eof()). Three measured failure modes on three inputs differing by one character. The hang is the one that costs the most time, because it does not look like a parsing bug.

Calling clear() and expecting the read to work next time. clear() resets flags and consumes nothing. Measured in Chapter 23.3 §1: clear() alone still spins forever.

Using the value after an unchecked read. It is 0, and 0 is also valid data.

Assuming eof() means "nothing left". §3 shows a stream with eof=1 whose last read succeeded and which converts to true.

std::endl in a loop. Measured ~17× for identical output.

Expecting written characters to be on disk. 2,000 characters written, 0 bytes in the file.

Reading a negative number into an unsigned and trusting the flags. 4294967295, fail=0.

14. Interview Relevance

The frozen curriculum marks this chapter High, and the question is usually a variant of "what is wrong with this loop?" over an eof()-controlled read. The answer that lands has three parts rather than one: it runs an extra iteration on a trailing newline and uses a fabricated value, it spins forever on a field that cannot be converted because a failed read consumes nothing, and the correct form puts the read in the condition so that continuing and succeeding are the same decision.

The follow-up is usually "what is the difference between fail() and bad()?" — fail() is failbit or badbit, bad() is badbit only, and operator bool is !fail() so it ignores eofbit. Being able to say that badbit is the unrecoverable one, and that failbit plus consuming the bad characters is recoverable, shows you have actually written a recovery path.

If you can add that std::endl is a flush and costs about an order of magnitude for identical bytes, you have said something most candidates have only heard as a style rule.

15. What You Should Be Able to Say Now

• That a stream is a sequence of characters plus four state bits, and that the console, a file and a string differ in nothing else that matters to the reading code.
• That operator bool is !fail(), so it ignores eofbit.
• That fail() is true when badbit is set, and bad() is the narrow, unrecoverable case.
• That a failed extraction writes 0 since C++11 and consumes nothing.
• Why while (in >> x) is correct by construction and while (!in.eof()) fails three different ways — measured on "1 2 3", "1 2 3\n" and "1 2 x".
• That characters are not on disk until a flush or close, that the block here is 4,096 bytes, and that std::endl costs about 17× a '\n' for the same file.
• That the formatting, not the file, is most of the per-line cost — 1.26×.

16. Exercises

Exercise 1 — the three inputs. Write the eof() loop and run it on "1 2 3", "1 2 3\n" and "1 2 x", with an iteration cap so the third one reports instead of hanging. Confirm the three outcomes. Then remove the cap and watch what the third one actually does.

Exercise 2 — the bits. Print all four bits after each step of §3's sequence and reproduce the transcript. Then add a step that calls in.clear() without consuming "abc" and read the next value; explain the result using §3's second observation.

Exercise 3 — the flush. Write 5,000 characters to a file in a loop, polling std::filesystem::file_size from the same program, and find the block size on your machine. Then write the same data with std::endl and with '\n' and compare the times and the file sizes.

Exercise 4 — the range problems. Read "-1" into an unsigned, "99999" into a short, and "3.7" into an int, printing the value and all four bits each time. Three different behaviours; say which are failures and which are silent conversions.

17. Summary

• A stream is a sequence of characters with state — a position and four bits — and the console, a file and a string are the same abstraction with different character sources. Write functions against std::istream&.
• The four bits: eofbit (a read went past the end), failbit (the operation did not do what was asked), badbit (the stream is broken), and goodbit meaning none of them. fail() is failbit or badbit; operator bool is !fail() and ignores eofbit.
• A failed extraction writes 0 and consumes nothing. Both halves matter: the value is indistinguishable from real data, and the position does not move.
• while (in >> x) is the loop. Measured, while (!in.eof()) agrees on "1 2 3", fabricates a zero-filled record on "1 2 3\n", and spins forever on "1 2 x" — a hang, not a wrong answer.
• The buffer is part of the model. 2,000 characters written, 0 bytes on disk; libc++ flushes in 4,096-byte blocks, so that is what a crash can lose.
• std::endl is '\n' plus a flush and costs a median 17× (range 12.6–30.9×) for byte-identical output. Use '\n' and flush deliberately.
• The destination is not the cost. The same lines to a file cost 1.26× the same lines to memory; the formatting is the rest.

Chapter 23.2 takes the output half of this model and makes it produce files that a human can read and a tool can parse — which turns out to depend on knowing which manipulators stick.