Case and Whitespace: upper, lower, strip

Part I · Module 4 · Chapter 4.7 · Case and Whitespace: upper, lower, strip

1. Two Comparisons That Should Work and Do Not

Chapter 3.4 taught ==, and on text it is exact: two strings are equal when they hold the same characters in the same order. That is the right definition, and it is also why this fails:

a = "PASS"
b = "pass"
print(a == b)

False

Both say the run passed. Python says they are different strings, and Python is correct — P is not p.

Here is the second failure, and it is worse because you cannot see the cause:

raw = "182.4\n"
print(raw == "182.4")

False

Nothing looks wrong. The two look identical on screen. They are not, and len() proves it:

raw = "182.4\n"
print(len(raw))
print(repr(raw))

6
'182.4\n'

Six characters, not five. There is a newline on the end, and repr() is what makes it visible — as Chapter 4.5 noted, repr() shows a value the way you would write it in source, so escapes appear as escapes instead of doing their job.

This chapter is the fix for both failures. The technique has a name — normalising — and it means putting text into a standard form before you compare it.

2. Case: lower() and upper()

The two methods read exactly as they are named:

print("PASS".lower())
print("pass".upper())

pass
PASS

Applied to the failing comparison, converting both sides to the same case makes the comparison ask the question you actually meant:

a = "PASS"
b = "pass"
print(a.lower() == b.lower())

True

Note carefully what this does and does not say. It does not make a and b equal — they are still different strings. It compares two new strings, both produced by lower(), which do hold the same characters.

3. Whitespace: strip()

strip() removes whitespace from the ends of a string:

line = "  WARNING: setup violation\n"
print(repr(line))
print(repr(line.strip()))

'  WARNING: setup violation\n'
'WARNING: setup violation'

Two leading spaces gone, trailing newline gone, and the text itself untouched. This single method is the most-used string operation in scripting, because almost every piece of text that comes from a file, a report or a command line has something clinging to its ends.

Two precise facts about what it removes.

It touches the ends only. Internal whitespace is left exactly as it is:

print(repr("  a   b  ".strip()))

'a   b'

The three spaces between a and b survive. This is worth being exact about, because "strip removes the spaces" is a common and wrong summary — it removes the spaces at the ends. If you need to do something about internal whitespace, that is a different job, and Chapters 4.8 and 4.9 together are the usual way to do it.

It removes whitespace of every kind, not just spaces. Newlines and tabs are whitespace too:

print(repr("\t\n  PASS \n\t".strip()))

'PASS'

Tabs, newlines and spaces all removed from both ends in one call. That is why strip() alone handles the invisible-newline problem from Section 1 — you do not need to know which kind of whitespace is there.

One end at a time

When you want only one end, there are two narrower methods. The l and r are left and right:

print(repr("  PASS  ".lstrip()))
print(repr("  PASS  ".rstrip()))

'PASS  '
'  PASS'

rstrip() is the one you will reach for most in practice: it removes the trailing newline from a line of text while leaving any leading indentation intact, which matters when the indentation is meaningful.

4. The Invisible Newline

This is the case the canonical curriculum singles out, and it deserves its own section because it explains a whole class of confusing bugs.

When text is written to a file or printed, each line is separated from the next by a newline character. That character is part of the line's text. So a line that displays as 182.4 is very often the six-character string "182.4\n".

You have not read a file yet — that is Module 15 — but the consequence is worth understanding now, because it is the reason a comparison against tool output fails for no visible reason:

raw = "182.4\n"
print(raw == "182.4")
print(raw.strip() == "182.4")

False
True

The first comparison is False and looks like a bug in your logic. It is not: the strings genuinely differ. The second is the same comparison after normalising, and it is True.

The length makes the difference concrete:

raw = "182.4\n"
print(len(raw))
print(len(raw.strip()))

6
5

This also matters for the number conversions in Chapter 2.11. int() and float() are tolerant of surrounding whitespace, but nothing else is — and stripping first costs you nothing.

5. strip(chars) Is Not Suffix Removal

Here is the trap, and it is the most misleading thing in this module.

strip() accepts an optional argument. It is tempting to read strip(".log") as "remove .log from the end", and on one example it appears to work:

print(repr("sim.log".strip(".log")))

'sim'

That looks like suffix removal. It is not. The argument is a set of characters to remove, not a piece of text to match. ".log" means "the characters ., l, o and g", and strip keeps removing any of those characters from both ends until it meets something else.

Watch it go wrong on a name that begins with one of those characters:

print(repr("log.sim.log".strip(".log")))

'sim'

It ate the front too. The leading log. was removed because l, o, g and . are all in the set. Two more, to make the behaviour unmistakable:

print(repr("axi_smoke".strip("axi")))
print(repr("xxaxi_smokexx".strip("x")))

'_smoke'
'axi_smoke'

The first removed a, x and i from the front — not the word axi, the characters. The second removed every x from both ends and stopped at a.

The honest tools

Python has two methods that really do what strip(".log") looks like it does. They match a whole piece of text, at one specified end, once:

print(repr("sim.log".removesuffix(".log")))
print(repr("log.sim.log".removesuffix(".log")))

'sim'
'log.sim'

The second result is the point: removesuffix left the front alone, because it removes a suffix and nothing else. Compare it against the strip(".log") result above — same input, different answer, and this one is the answer you wanted.

removeprefix is the mirror image:

print(repr("UVM_ERROR: x".removeprefix("UVM_ERROR: ")))

'x'

That is the same job Chapter 4.5 did with full[len("UVM_ERROR: "):], said more directly. Both are fine; this one is harder to get wrong.

6. Every One of These Returns a New String

Chapter 4.1 stated this and it is worth demonstrating again here, because this is the chapter where the mistake actually costs you something:

line = "  PASS  "
clean = line.strip()
print(repr(line))
print(repr(clean))

'  PASS  '
'PASS'

line is unchanged. clean is the new, stripped string. The method produced a value and the assignment is what kept it.

Which means this does nothing at all:

line = "  PASS  "
line.strip()
print(repr(line))

'  PASS  '

The stripped string was created and immediately discarded, because nothing was done with it. Chapter 4.11 is entirely about why Python works this way, and how to spot the mistake in your own code.

Because each method returns a string, calls can be chained — the result of one becomes the value the next works on:

print(repr("  WARNING \n".strip().lower()))

'warning'

Read it left to right: take the string, strip it, then lowercase the result. That is the normalising idiom in its usual form, and it is how you would prepare a line before comparing it:

t1 = "  Pass \n"
print(t1.strip().lower() == "pass")

True

Both failures from Section 1 are now fixed by one expression.

7. The Neighbours

Three more case methods exist. They are listed here as a summary rather than taught, because they are variations on what Section 2 already established:

For comparing English-language tool output, lower() is the one to use and the others are rarely needed. Do not use title() on data you intend to compare — it changes the text in ways that depend on where the spaces are.

8. Predict, Run, Observe

Predict all six results before reading on. Two of them are the traps from this chapter.

line = "  FAIL: hold  \n"
print(repr(line.strip()))
print(repr(line.rstrip()))
print(repr(line.strip().lower()))
print(len(line.strip()))
print(repr("  a  b  ".strip()))
print(repr("report.txt".strip(".txt")))

The verified output:

'FAIL: hold'
'  FAIL: hold'
'fail: hold'
10
'a  b'
'repor'

The fifth keeps its internal spaces — strip() works on the ends only.

The sixth is the trap, and it is worse than the .log example. strip(".txt") removes the characters ., t and x from both ends, so after taking .txt off the end it carries on and removes the t of report as well, leaving repor. If you predicted report you read the argument as a suffix. "report.txt".removesuffix(".txt") is the call that gives report.

9. Exercises

Exercise 1 — make the comparison work. Each of these is False. For each, write a version that is True by normalising both sides, and say what was different about the two strings: (a) "PASS" == "pass" (b) "182.4\n" == "182.4" (c) " OK " == "OK" (d) "Fail\n" == "fail"

Exercise 2 — predict the repr. Without running anything, give the exact result of each, then verify with repr(): (a) " x ".strip() (b) " x ".lstrip() (c) "\n\nx".strip() (d) " a b ".strip() (e) "".strip()

Exercise 3 — the invisible character. A colleague says runtime == "182.4" is False even though printing runtime shows 182.4. Write the two lines of diagnosis you would ask them to run, and say what you expect each to reveal.

Exercise 4 — the thrown-away result. This prints the unchanged string. Explain in one sentence what happened to the stripped value, and fix it:

status = "  FAIL  \n"
status.strip()
print(repr(status))

Exercise 5 — prove the trap. Run both of these and write down the two results. Then explain in two sentences why they differ, and which one you would use in a script:

print("sim.log.log".strip(".log"))
print("sim.log.log".removesuffix(".log"))

Exercise 6 — pick the method. For each task, name the method you would use and say why the obvious alternative is wrong: (a) remove the newline from the end of a line but keep its leading indentation (b) remove the extension from report.txt (c) compare two status words whose case you do not control (d) remove the UVM_ERROR: marker from the front of a line

Exercise 7 — chain it. Write one expression that takes " Setup Violation \n" and produces exactly setup violation. Then say what order the two methods ran in, and whether swapping them would change the answer.

10. Summary

• == on text is exact, which is why "PASS" == "pass" and "182.4\n" == "182.4" are both False.
• Normalising means putting text into a standard form before comparing it. lower() for case, strip() for whitespace.
• Normalise both sides, not just the one you happen to be looking at.
• strip() removes whitespace from the ends only. Internal spaces survive — "strip removes the spaces" is not an accurate summary.
• strip() removes whitespace of every kind — spaces, tabs and newlines — so you need not know which is there.
• lstrip() and rstrip() do one end. rstrip() is the usual choice for removing a trailing newline.
• Text from a file, report or command carries whitespace you did not add, including a trailing newline. Strip it before comparing.
• print(repr(value)) and print(len(value)) diagnose an invisible character in one line each.
• strip(chars) takes a set of characters, not a suffix. "report.txt".strip(".txt") is "repor", not "report".
• removeprefix() and removesuffix() match a whole piece of text at one end, once. They are what strip(".log") looks like it does. Python 3.9 and later.
• Every one of these returns a new string. A method call on a line by itself changes nothing.
• Methods chain: line.strip().lower() reads left to right.

Normalised text is text you can trust. The next chapter takes a normalised line apart into its separate fields.