Numeric Conversion and Comparison

Part II · Module 8 · Chapter 8.5 · Numeric Conversion and Comparison

1. Why This Matters

awk has no declared types, so every comparison has to decide at run time whether it is comparing numbers or text. The decision is made from the origin of each value, not from what it looks like, and that is why it is hard to predict until you know the rule.

It is rated critical because the failure is a plausible wrong answer rather than an error. A threshold that lets the wrong rows through, a maximum that picks the wrong record, a classification that marks an unscored test as passing — all from a comparison that is doing exactly what it was told.

2. The Rule

Two definitions and one rule.

A number is a numeric literal or the result of arithmetic. A numeric string is a value that came from outside the program — a field, a -v assignment, an ENVIRON entry, a getline result — whose text looks like a number.

The comparison is numeric if both sides are numbers or numeric strings. Otherwise it is a string comparison.

The consequence that catches everyone: a quoted constant is a string, not a numeric string, even when it contains digits. So comparing a field against 9 is numeric and against "9" is textual.

Sections 3 to 7 demonstrate each row.

3. Two Fields Compare Numerically

10 9
9 10

awk '{ printf "%s vs %s -> $1 < $2 is %d\n", $1, $2, ($1 < $2) }' pair.txt

10 vs 9 -> $1 < $2 is 0
9 vs 10 -> $1 < $2 is 1

Correct arithmetic in both directions. Both sides are fields, both look numeric, so the comparison is numeric.

This is the common case and it is why the rule goes unnoticed for a long time — comparing two columns of a report just works.

4. A Field Against a Number Is Numeric

alu 10
fifo 9
uart 100

awk '{ printf "%-6s %3s  $2 < 9: %d\n", $1, $2, ($2 < 9) }' nums.txt

alu     10  $2 < 9: 0
fifo     9  $2 < 9: 0
uart   100  $2 < 9: 0

None is less than 9. Correct — 9 is a numeric literal, so both sides are numeric.

5. A Field Against a String Constant Is Text

One character changes, and so does the answer:

awk '{ printf "%-6s %3s  $2 < \"9\": %d\n", $1, $2, ($2 < "9") }' nums.txt

alu     10  $2 < "9": 1
fifo     9  $2 < "9": 0
uart   100  $2 < "9": 1

10 is less than "9", and so is 100.

As text, "10" and "100" both begin with 1, which sorts before 9. The comparison is correct for the question it was asked; it was asked the wrong question.

The same applies with the quoted value on the left:

awk 'BEGIN { printf "\"10\" < 9  : %d   (string)\n10 < 9      : %d   (numeric)\n", ("10" < 9), (10 < 9) }' </dev/null

"10" < 9  : 1   (string)
10 < 9      : 0   (numeric)

"10" < 9 is true. The quoted constant forces a string comparison even though the other side is a number.

6. What Counts as a Numeric String

A field is a numeric string if its text looks like a number, and awk's idea of that is wider than decimal digits.

Scientific notation:

p 1e3
q 999

awk '{ printf "%-3s [%s]  $2 > 999: %d\n", $1, $2, ($2 > 999) }' sci.txt

p   [1e3]  $2 > 999: 1
q   [999]  $2 > 999: 0

1e3 was recognised as 1000 and compared numerically. Had it been treated as text, "1e3" > "999" would be false — 1 sorts before 9.

Hexadecimal, on this awk:

r 0x10
s 15

awk '{ printf "%-3s [%s]  $2 > 15: %d\n", $1, $2, ($2 > 15) }' hex.txt

r   [0x10]  $2 > 15: 1
s   [15]  $2 > 15: 0

0x10 compared as 16. Chapter 5.3 section 9 found the same in arithmetic and flagged it as implementation-specific — gawk in POSIX mode converts "0x10" to 0. The portable position is unchanged: do not rely on it, and use Chapter 5.3's positive guard so a hex value is rejected rather than interpreted either way.

What is not a numeric string is anything whose text does not parse as a number — n/a, VIOLATED, 1.5ns. Those force a string comparison, which is Chapter 5.4 section 9's finding: an unscored test compares as PASS because "n/a" >= "80" is true as text.

7. Forcing the Comparison

Two idioms, one for each direction. On the same field:

awk 'NR == 1 { printf "raw   $2 < 9    : %d\n+ 0   $2 + 0 < 9: %d\n\"\"    $2 \"\" < \"9\": %d\n", ($2 < 9), ($2 + 0 < 9), ($2 "" < "9") }' nums.txt

raw   $2 < 9    : 0
+ 0   $2 + 0 < 9: 0
""    $2 "" < "9": 1

+ 0 forces numeric. Adding zero produces a number, and a number on one side with a numeric string on the other is a numeric comparison.

"" forces string. Concatenating an empty string produces an ordinary string, which strips the numeric-string status — and that is the only way to get a deliberate text comparison of two fields.

Both are cheap and both say what they mean. The habit worth forming: when a comparison involves anything from outside the program, force the kind you want. $2 + 0 > limit + 0 is four characters longer than $2 > limit and cannot be wrong.

A sorting comparison needs the same care, and Chapter 7.7 section 6 made this point about sorting array keys by hand: key[j] < key[i] is numeric or textual depending on the keys, so force it.

8. The Classic Symptom

What an accidental string comparison looks like in a report:

10
9
100

awk 'NR == 1 { m = $1 } $1 + 0 > m + 0 { m = $1 } END { print "max numeric:", m }' s.txt

max numeric: 100

awk 'NR == 1 { m = $1 } $1 "" > m "" { m = $1 } END { print "max string:", m }' s.txt

max string: 9

A maximum of 9 from a column containing 100. As text, "9" is the largest of the three because it begins with the highest digit.

This is the signature to memorise. A worst-case slack, a peak runtime or a largest cell count that comes back as a single-digit value from a column containing larger ones is almost always this — and it is distinguishable from the seeding bug in Chapter 5.1 section 9, which produces an empty answer rather than a small one.

Two symptoms, two causes:

9. Where This Already Appeared

The rule has been visible in five earlier chapters, and collecting them makes it concrete:

• Chapter 4.2 — $1 > 9 is true and $1 > "9" is false for a field holding 10. Section 5's finding, met first as a filtering surprise.
• Chapter 5.1 §3 and Exercise 6 — a field holding 0 is false while the constant "0" is true. Truthiness is the one-sided version of the same rule: a string is tested for emptiness and a number for zero.
• Chapter 5.4 §9 — if ($2 >= 80) classifies an unscored test as PASS, because n/a is not a numeric string so the comparison is textual and n sorts above 8. The direction even depends on the placeholder's spelling.
• Chapter 6.1 §3 — array subscripts are converted to strings, so a[1] and a["1"] are one element while "01" is another.
• Chapter 8.4 §4 — a missing field equals "" and does not equal 0, because "" == 0 is a string comparison between "" and "0".

One rule, five faces. That is the argument for learning it rather than memorising the symptoms.

10. Common Mistakes

Quoting a numeric threshold. $2 < "9" is textual and disagrees with $2 < 9 — section 5.

Comparing against a concatenated value. Concatenation always yields a string.

Testing on single-digit data. A string comparison is correct on one digit and wrong on two.

Assuming a maximum that looks small is a seeding bug. Empty means seeding; small means string comparison — section 8.

Relying on hex conversion. It works here and not everywhere — section 6.

Expecting "" == 0 to be true. It is a string comparison and false — Chapter 8.4.

Using a bare if (v) on a field. It asks "non-zero?" when you probably meant "present?" — Chapter 5.1.

Hand-sorting keys without forcing the comparison. The kind depends on the keys — Chapter 7.7 §6.

Forcing with + 0 on a value that should stay text. It converts VIOLATED to 0 silently; validate first — Chapter 5.3 §11.

11. Exercises

Exercise 1 — Predict the output

awk 'BEGIN { printf "%d %d\n", ("10" < 9), (10 < 9) }' </dev/null

1 0

The quoted constant forces a string comparison, where "10" sorts before "9".

Exercise 2 — Field against number and string

Run both comparisons on the same data and explain the difference.

alu 10
fifo 9
uart 100

awk '{ printf "%-6s %3s  numeric: %d   string: %d\n", $1, $2, ($2 < 9), ($2 < "9") }' nums.txt

alu     10  numeric: 0   string: 1
fifo     9  numeric: 0   string: 0
uart   100  numeric: 0   string: 1

Only fifo agrees, because 9 is a single digit and the two orderings coincide for it.

Exercise 3 — Debug a maximum of 9

A peak-value report returns 9 from a column whose largest entry is 100.

10
9
100

awk 'NR == 1 { m = $1 } $1 "" > m "" { m = $1 } END { print "max:", m }' s.txt

max: 9

The "" concatenation strips the numeric-string status from both sides, so the comparison is textual and "9" is the largest.

awk 'NR == 1 { m = $1 } $1 + 0 > m + 0 { m = $1 } END { print "max:", m }' s.txt

max: 100

Exercise 4 — Force both ways

Show the three forms on one field.

awk 'NR == 1 { printf "raw: %d   +0: %d   \"\": %d\n", ($2 < 9), ($2 + 0 < 9), ($2 "" < "9") }' nums.txt

raw: 0   +0: 0   "": 1

Raw and + 0 agree because the field is already a numeric string; "" is the only one that changes the operation.

Exercise 5 — Scientific notation

Confirm that a field in exponent form is compared numerically.

p 1e3
q 999

awk '{ printf "%-3s [%s] > 999: %d\n", $1, $2, ($2 > 999) }' sci.txt

p   [1e3] > 999: 1
q   [999] > 999: 0

1e3 is 1000. A text comparison would have made it false, since 1 sorts below 9.

Exercise 6 — Engineering application

Write a slack filter that cannot be fooled: validate each value, force the comparison, and report what it excluded.

path_a 0.5
path_b -0.2
path_c n/a
path_d -1.25

awk -v limit=0 '$2 !~ /^-?[0-9]+(\.[0-9]+)?$/ { bad++; next } $2 + 0 < limit + 0 { printf "%-8s %8.3f VIOLATED\n", $1, $2 + 0; v++ } { n++ } END { printf "(%d of %d usable path(s) violate slack < %s", v + 0, n, limit; if (bad) printf "; %d unusable", bad; printf ")\n" }' slack.rpt
echo "status: $?"

path_b     -0.200 VIOLATED
path_d     -1.250 VIOLATED
(2 of 3 usable path(s) violate slack < 0; 1 unusable)
status: 0

Three things make this safe. The positive validation rejects n/a before it reaches a comparison — which Chapter 5.4 section 9 showed would otherwise classify it by its first character. The forced comparison $2 + 0 < limit + 0 is immune to limit arriving as a string. And the counts state what was used and what was not.

Dropping the validation shows what it was protecting against:

awk -v limit=0 '$2 + 0 < limit + 0 { printf "%-8s VIOLATED\n", $1; v++ } { n++ } END { printf "(%d of %d)\n", v + 0, n }' slack.rpt

path_b   VIOLATED
path_d   VIOLATED
(2 of 4)

Here + 0 turned n/a into 0, which is not less than 0, so it was silently counted as a passing path — the denominator is 4 and one of those four was never measured. The validated version reports 3 usable and names the exclusion.

Exercise 7 — Challenge: detect an accidental string comparison

Write a check that reveals whether a comparison in a script is numeric or textual, without reading the script.

printf '10\n9\n100\n' > probe.txt
for mode in numeric string; do
  printf "%-8s max: " "$mode"
  if [ "$mode" = numeric ]; then
    awk 'NR == 1 { m = $1 } $1 + 0 > m + 0 { m = $1 } END { print m }' probe.txt
  else
    awk 'NR == 1 { m = $1 } $1 "" > m "" { m = $1 } END { print m }' probe.txt
  fi
done

numeric  max: 100
string   max: 9

The probe works because 10, 9 and 100 order differently under the two rules — numerically 9 < 10 < 100, textually "10" < "100" < "9". Any three values with that property will do; three single-digit values will not.

That is the transferable technique: to find out which comparison a script performs, feed it data on which the two answers differ. A column of single-digit values cannot distinguish them, which is precisely why the bug survives testing — and it is the same argument Chapter 7.7 section 7 made about PROCINFO, where a two-key test could not tell a working sort from an inert one.

Applying it to a script you did not write:

awk 'NR == 1 { m = $1 } $1 > m { m = $1 } END { print "unforced max:", m }' probe.txt

unforced max: 100

100, so that comparison is numeric — the fields are numeric strings and nothing stripped them. The same script given a column containing a non-numeric value would behave differently, which is why validation belongs before the comparison rather than after it.

12. Summary

• A comparison is numeric if both sides are numbers or numeric strings; otherwise it is textual.
• A numeric string comes from outside the program — a field, -v, ENVIRON, getline — and looks like a number.
• A quoted constant is a string, so $2 < "9" is textual while $2 < 9 is numeric.
• Two fields compare numerically, which is why the rule goes unnoticed.
• 1e3 is a numeric string. 0x10 is too on this awk, and that part is implementation-specific.
• A value whose text is not a number forces a text comparison — the cause of Chapter 5.4's unscored test passing.
• + 0 forces numeric; "" forces string. Use one deliberately on anything from outside.
• An empty maximum means a seeding bug; a small maximum means a string comparison.
• Single-digit test data cannot distinguish the two rules, which is how the bug ships.
• Validate before forcing, or + 0 converts a malformed value to a confident zero.

That is the last of the semantic traps. Chapter 8.6 — POSIX awk, gawk, and Portability turns to a different kind of reliability: which of the features this track has used are standard and which are extensions, and why the flag that is supposed to enforce the distinction does nothing at all on the awk used throughout these chapters.