How Perl Creates Nested Data Automatically

Part I · Module 7 · Chapter 7.9 · How Perl Creates Nested Data Automatically

1. The Subscript Decides the Type

use strict;
use warnings;

my %a;
$a{alu}[0] = "clk";

my %h;
$h{alu}{slack} = -0.12;

my %deep;
$deep{a}{b}[0]{c} = "bottom";

printf "after \$a{alu}[0]      : a{alu} is a %s ref\n", ref($a{alu});
printf "after \$h{alu}{slack}  : h{alu} is a %s ref\n", ref($h{alu});
printf "after \$deep{a}{b}[0]{c}:\n";
printf "   deep{a}       is a %s ref\n", ref($deep{a});
printf "   deep{a}{b}    is a %s ref\n", ref($deep{a}{b});
printf "   deep{a}{b}[0] is a %s ref\n", ref($deep{a}{b}[0]);
printf "   the value     is %s\n", $deep{a}{b}[0]{c};

after $a{alu}[0]      : a{alu} is a ARRAY ref
after $h{alu}{slack}  : h{alu} is a HASH ref
after $deep{a}{b}[0]{c}:
   deep{a}       is a HASH ref
   deep{a}{b}    is a ARRAY ref
   deep{a}{b}[0] is a HASH ref
   the value     is bottom

Three facts, and the third is the striking one.

The subscript that follows decides the type. $a{alu}[0] put an array reference in $a{alu}, because the next thing was [0]. $h{alu}{slack} put a hash reference there, because the next thing was {slack}. Perl is not guessing from context or from a declaration — it reads the subscript and builds what that subscript needs.

It applies at every level independently. $deep{a}{b}[0]{c} produced hash, then array, then hash, following {b}, [0], {c} in order.

And it happens in one statement. That single assignment created three containers and stored a value at the bottom. Writing it out by hand would be four statements and two constructors:

my %deep;
$deep{a}         = {};
$deep{a}{b}      = [];
$deep{a}{b}[0]   = {};
$deep{a}{b}[0]{c} = "bottom";

That is what autovivification saves you, and why every nested build in Chapter 7.7 and Chapter 7.8 was a single line.

2. Where It Happens

Five contexts, all measured in one run on a fresh empty hash.

use strict;
use warnings;

my %h;

push @{ $h{p} }, 1;
printf "1. push        : p created %s (%s)\n", (exists $h{p}?"YES":"no"), (ref($h{p})||"none");

for (@{ $h{f} }) { }
printf "2. foreach     : f created %s (%s)\n", (exists $h{f}?"YES":"no"), (ref($h{f})||"none");

my $e = exists $h{e}{k};
printf "3. exists      : e created %s (%s)\n", (exists $h{e}?"YES":"no"), (ref($h{e})||"none");

$h{w}{deep} = 1;
printf "4. assignment  : w created %s (%s)\n", (exists $h{w}?"YES":"no"), (ref($h{w})||"none");

my $v = $h{r}{deep};
printf "5. nested read : r created %s (%s)\n", (exists $h{r}?"YES":"no"), (ref($h{r})||"none");

printf "top-level keys now: %d  [%s]\n", scalar(keys %h), join(" ", sort keys %h);

1. push        : p created YES (ARRAY)
2. foreach     : f created YES (ARRAY)
3. exists      : e created YES (HASH)
4. assignment  : w created YES (HASH)
5. nested read : r created YES (HASH)
top-level keys now: 5  [e f p r w]

All five created a key. The hash started empty and ended with five.

Cases 1 and 4 are the feature working as intended — you were building something. Cases 2, 3 and 5 are the ones to know about:

• foreach over a missing list creates it, because foreach aliases its list elements and aliasing requires something to alias to. The loop then runs zero times, which is why Chapter 7.3 §7 found that a misspelled key produces silence rather than an error.
• exists at depth creates the level above, which Chapter 7.8 §4 measured in detail.
• A nested read creates the level above too — my $v = $h{r}{deep} made $h{r} a hash reference even though nothing was assigned into the structure.

3. The Precise Boundary

Case 5 above deserves its own measurement, because the rule is finer than "reading creates".

use strict;
use warnings;

my %h;

my $v = $h{a}{b};

printf "a exists      : %s\n", (exists $h{a} ? "YES" : "no");
printf "a is a        : %s ref\n", (ref($h{a}) || "none");
printf "b exists      : %s\n", (exists $h{a}{b} ? "YES" : "no");
printf "fields in a   : %d\n", scalar(keys %{ $h{a} });
printf "the value read: %s\n", (defined $v ? $v : "undef");

a exists      : YES
a is a        : HASH ref
b exists      : no
fields in a   : 0
the value read: undef

The level above was created; the leaf was not. $h{a} is now a real, empty hash reference. $h{a}{b} does not exist, and the value read back was undef.

The reason is the rule from §1 restated: Perl needed a container in $h{a} so it could apply {b} to it, and $h{a} was a writable position, so it built one. It did not need to create $h{a}{b} — nothing was subscripting that, so undef was a perfectly good answer.

So the rule is: every level that gets subscripted is created; the final level being read is not. A four-level read creates three containers.

4. Not Creating Things

The portable defence is short-circuit evaluation, guarding from the outside in.

use strict;
use warnings;

my %h = ( alu => { slack => -0.12 } );

printf "before            : %d key(s)  [%s]\n", scalar(keys %h), join(" ", sort keys %h);

for my $blk (qw(alu fifo regf)) {
    my $has = exists $h{$blk} && exists $h{$blk}{slack};
    printf "  %-5s has slack : %s\n", $blk, ($has ? "yes" : "no");
}

printf "after guarded     : %d key(s)  [%s]\n", scalar(keys %h), join(" ", sort keys %h);

before            : 1 key(s)  [alu]
  alu   has slack : yes
  fifo  has slack : no
  regf  has slack : no
after guarded     : 1 key(s)  [alu]

One key before, one key after, having probed three block names including two that do not exist. && never evaluates its right side once the left is false, so the second level is never reached. That is Chapter 3.8's short-circuit behaviour being load-bearing rather than merely convenient.

The same shape extends to any depth:

if (exists $h{a} && exists $h{a}{b} && exists $h{a}{b}{c}) { ... }

It gets verbose, which is a fair criticism, and there are two reasonable ways to shorten it:

• Hold the intermediate reference once when you are about to use it several times:

my $rec = $h{$blk} or next;        # skips the block entirely if absent
printf "%s\n", $rec->{slack} // "(none)";

• Decide you do not care. If your program never counts or iterates the top-level keys, a few spurious empty records are invisible. Be deliberate about that rather than discovering it.

5. Why This Matters in Engineering Work

It is why the parsing idioms are one line. push @{ $by_block{$blk} }, $port and $report{$blk}{$field} = $value both depend on it entirely. Understanding the mechanism is what lets you trust those lines rather than defensively initialising around them.

It makes "is this missing" a question you must ask carefully. Checking whether a block has a field can add the block. In a script that then reports how many blocks it found, the count is wrong and nothing failed. That is the bug this chapter exists to prevent.

And it means silence is not evidence. A loop that runs zero times, a count that comes back zero, a lookup that returns undef — none of those distinguish "the data says nothing" from "I spelled the key wrong". When the distinction matters, test the key.

6. Common Mistakes

Testing a nested key without a guard. exists $h{a}{b} creates $h{a}. Guard with &&.

Trusting a top-level key count after probing optional fields. The probes may have added keys.

Assuming a read cannot change anything. A nested read creates every level it subscripts.

Assuming a missing structure will announce itself. A foreach over it runs zero times silently; a count in an argument list returns 0.

Expecting scalar(@{ $h{missing} }) to behave the same everywhere. It dies as an rvalue and returns 0 as a function argument.

Reaching for no autovivification. Not core, and often not installed. Use the guard.

Initialising intermediate levels "to be safe". Unnecessary, and it hides the fact that you expected them to exist.

7. Exercise

Exercise 1 — the subscript decides. On two separate empty hashes, write $a{x}[0] = 1 and $h{x}{k} = 1. Print ref() of $a{x} and $h{x}. State in one sentence what Perl used to decide.

Exercise 2 — four levels, one statement. Write $d{a}{b}[0]{c} = "bottom" on an empty hash. Print ref() at each of the three intermediate levels and the value at the bottom. Then write the equivalent as separate statements and count them.

Exercise 3 — the five contexts. Reproduce §2 exactly. Report which contexts created a key and the final top-level key count. Then say which two of the five surprised you.

Exercise 4 — the boundary. Run my $v = $h{a}{b}; on an empty hash. Report whether a exists, what ref($h{a}) is, whether b exists, how many fields a has, and what $v is. State the rule in one sentence.

Exercise 5 — context decides. Write my $n = scalar(@{ $h{missing} }); and record STDERR and the exit status. Then write printf "count %d\n", scalar(@{ $h{missing} }); and record the output, the exit status, and whether the key was created. Explain why the container is not the reason they differ.

Exercise 6 — the guard. Starting from a one-key hash, probe three block names for a nested field using exists $h{$b} && exists $h{$b}{slack}. Print the key count before and after. Then remove the guard and print the count again.

Exercise 7 — hold the reference. Rewrite Exercise 6's loop using my $rec = $h{$blk} or next; and confirm the key count is unchanged. Say which version you would put in a shared script and why.

Exercise 8 — the pragma. Try use autovivification;. Record what happens on your machine. If it is missing, write one sentence on why a correctness-critical script should not depend on it.

8. Summary

• Autovivification creates a missing intermediate level when it is subscripted in a position Perl may assign to.
• The subscript decides the type: [0] creates an array reference, {k} creates a hash reference.
• It applies at every level independently, so one assignment can build a four-level structure.
• Every level that gets subscripted is created; the final level being read is not. Reading $h{a}{b} makes $h{a} an empty hash and leaves $h{a}{b} absent.
• push, foreach, exists at depth, assignment and a nested read all create. Measured: five contexts, five keys.
• The same dereference dies as an rvalue and autovivifies as a function argument — my $n = scalar(@{ $h{missing} }) is fatal, the same expression inside printf returns 0 and creates the key. The surrounding position decides, not the container.
• Guard with && from the outside in. Measured: one key before, one after, probing three names.
• no autovivification is a CPAN module, not core, and is not installed here. The guard needs nothing.
• Silence is not evidence. Zero iterations and a zero count do not distinguish empty data from a misspelled key.

Chapter 7.10 uses all of this to walk a structure of unknown shape, dispatching on what each node turns out to be — which needs a function that can call itself and a careful eye on one string-interpolation trap.