Speaker A
What does it mean for a file to live somewhere? It's one of those phrases that just comes out automatically. You point at a file path and you say, "That's where that file lives." And nobody ever stops on that sentence.
Explore how file paths work as routes rather than static addresses, revealing the clever filesystem structure and path resolution.
Key Takeaways
- File paths function as routes or instructions, not fixed addresses.
- Each directory contains a table mapping names to inode numbers, enabling efficient path resolution.
- Path resolution happens step-by-step, validating each component sequentially.
- Renaming directories only requires updating a single directory entry, not all nested paths.
- The shell and filesystem work together to interpret and resolve paths dynamically.
What the video covers
- File paths are commonly understood as locations where files live, but this video explores the deeper meaning behind this phrase.
- A file path is not a static address but a set of sequential instructions or a route to reach a file step-by-step.
- Paths are resolved one directory at a time, starting from either the root or the current directory, using components like '.' and '..'.
- The filesystem does not store full paths in one place; instead, each directory holds a small table mapping names to inode numbers.
- This step-by-step resolution is called a 'walk', which explains why paths must be valid at each step to succeed.
- Absolute paths start with a slash and relative paths start from the current directory, but both follow the same resolution process.
- Renaming a directory only changes one entry in one directory's table, making filesystem updates efficient.
- The video also touches on symbolic links, file metadata, and how the shell interacts with paths and the working directory.
- It clarifies common misconceptions about paths being simple strings or addresses and explains the underlying system design.
- The filesystem's structure allows flexible navigation, linking, and efficient management of files and directories.
Chapters
- 00:00What does it mean for a file to live somewhere?
- 01:16Understanding the slash as a separator
- 02:08Paths as routes rather than addresses
- 04:17Path meandering and step-by-step resolution
- 05:18Meaning of being somewhere in the filesystem
- 06:18Testing paths and shell behavior
- 07:12Filesystem tables and directory entries
- 09:18Step-by-step path resolution called a walk
- 09:31Special directory entries: dot and dot-dot
- 10:20File metadata, renaming, and linking
Full Transcript — Download SRT & Markdown
Speaker A
[music] because it sounds so simple. But maybe there's something here worth pulling apart. So, let's take this phrase seriously for a second and see how far it can take us.
Speaker A
[music] Because it sounds so simple. But maybe there's something here worth pulling apart. So, let's take this phrase seriously for a second and see how far it can take us.
Speaker A
The thing is there when you arrive. And it's only there. A thing can't be in two places at once.
Speaker A
If something lives somewhere, then that somewhere is a place. And places have properties. You can go to a place.
Speaker A
To see what's going on, let's start with the string itself. The simple interpretation is that the string is one big identifier. Sort of a coordinate that points at some spot on disk, the same way that a street address
Speaker A
The thing is there when you arrive. And it's only there. A thing can't be in two places at once.
Speaker A
The slash in the middle is just a separator to tell us where one name ends and the next begins.
Speaker A
All of those sound like safe assumptions, but we'll see just how many of them actually apply by the end.
Speaker A
Bin is another directory sitting inside of the root directory. And zsh is a file inside of bin.
Speaker A
To see what's going on, let's start with the string itself. The simple interpretation is that the string is one big identifier. Sort of a coordinate that points at some spot on disk, the same way that a street address
Speaker A
It's helpful to see this string less like an address and more like a set of sequential instructions read one at a time from left to right.
Speaker A
points at a house. If we look closer, though, the string breaks into three parts. There's a leading slash, then there's bin, and then zsh.
Speaker A
And if it really is a set of instructions, and if each step is resolved on its own, then a path should be able to meander a bit.
Speaker A
The slash in the middle is just a separator to tell us where one name ends and the next begins.
Speaker A
And as long as every individual step is valid, the whole operation should work. In order to test that though, we need two more pieces. We need a way to refer to where we currently are, and a way to
Speaker A
The leading slash is how we refer to the root directory, which is the topmost directory in the file system hierarchy.
Speaker A
Two dots means one directory above. For example, from bin, two dots will take us back to the root.
Speaker A
Bin is another directory sitting inside of the root directory. And zsh is a file inside of bin.
Speaker A
We'll use the version flag so that we actually get some output. First with the program's name, and then with our meandering path.
Speaker A
Zsh is a program file in this case, although the system doesn't much care what type of file it is.
Speaker A
So the fact that it works tells us that paths are routes rather than addresses.
Speaker A
It's helpful to see this string less like an address and more like a set of sequential instructions read one at a time from left to right.
Speaker A
At this point, you might suspect that the shell is responsible for tidying up this string before anything else reads it. Perhaps by manually collapsing the dots down into a reasonable path, and then passing that along.
Speaker A
We start from root. And then from there, we go into bin. And then inside bin, we find zsh.
Speaker A
Except, we can't. In this example, there is no directory called fake. And if the shell were doing string math, then this wouldn't matter.
Speaker A
And if it really is a set of instructions, and if each step is resolved on its own, then a path should be able to meander a bit.
Speaker A
So when fake turns up missing, there's nowhere to continue from and the path breaks.
Speaker A
It should be able to go in somewhere, come back out, head off somewhere else, and come back again.
Speaker A
Thus far, all of these paths have begun at the root directory, but they don't all have to begin there.
Speaker A
And as long as every individual step is valid, the whole operation should work. In order to test that though, we need two more pieces. We need a way to refer to where we currently are, and a way to
Speaker A
Paths beginning at the root directory, by contrast, are called absolute paths. This often gets taught as though there are two different types of paths with fundamentally different behavior.
Speaker A
go back out. A single dot means this directory. Wherever we happen to be right now, a single dot refers back to that spot.
Speaker A
The only difference is where the path begins, and that's decided only by whether the first character is a slash.
Speaker A
Two dots means one directory above. For example, from bin, two dots will take us back to the root.
Speaker A
Like, what does it even mean to be somewhere in the file system? I'll have to come back to that question later because it's a cleaner explanation with a few more pieces in place.
Speaker A
Together, we can use these two to define a meandering path to zsh. And we can verify that this works by calling the zsh program with this meandering path.
Speaker A
It looks obvious enough with a pretty animated tree on screen, but there's no tree built into the computer.
Speaker A
We'll use the version flag so that we actually get some output. First with the program's name, and then with our meandering path.
Speaker A
And somewhere between that and /bin/csh, there sits a system that can turn an arbitrary human-readable name into the right bytes every time.
Speaker A
And they produce the same output. This little trick actually tells us something meaningful, because if a path were a static address, then this string would be meaningless.
Speaker A
So, before I show you how it's done, have a think about how you might do it yourself.
Speaker A
So the fact that it works tells us that paths are routes rather than addresses.
Speaker A
That could work, and it'd be fast, too. But, think about what happens under this model if you rename just a single folder.
Speaker A
/bin/zsh is a route, too. It's a rather short and sensible one, but it's a route all the same.
Speaker A
This means that the table would also store lots of redundant information, which is less than optimal.
Speaker A
At this point, you might suspect that the shell is responsible for tidying up this string before anything else reads it. Perhaps by manually collapsing the dots down into a reasonable path, and then passing that along.
Speaker A
We'll give the whole disk to the root directory and let each nested directory subdivide its parent's region.
Speaker A
And if that were true, then we should be able to trick the shell by inserting an invalid destination in the middle of our path, like so.
Speaker A
And if you get it wrong, you're back to shifting everything around and making hundreds of thousands of edits for a simple operation.
Speaker A
Except, we can't. In this example, there is no directory called fake. And if the shell were doing string math, then this wouldn't matter.
Speaker A
We can do better. Neither of these models is wrong about everything. The tree is the obvious conceptual model. It just can't be carved onto the disk directly.
Speaker A
This part of the string cancels out and we'd land at /bin/zsh regardless. It fails though because every single name, including the dots, gets looked up in order from where the last one landed.
Speaker A
The way we reconcile these two models is pretty clever. Instead of one enormous table for the whole disk, we'll give each directory a small table of its own with the name on the left and a number on the right
Speaker A
So when fake turns up missing, there's nowhere to continue from and the path breaks.
Speaker A
Nothing anywhere records a full path because nothing has to. Each directory knows only of its own contents.
Speaker A
We'll term this step-by-step resolution a walk. It stands to reason that needs to begin somewhere.
Speaker A
And this is why the walk has to happen one step at a time. There's no table anywhere that can resolve {slash} bin {slash} zsh in one shot because that information doesn't exist in one place.
Speaker A
Thus far, all of these paths have begun at the root directory, but they don't all have to begin there.
Speaker A
Let's go ahead and take a look at one. The command for this is called ls, which is short for list.
Speaker A
They can also begin from wherever you currently are, like so. This is called a relative path because it describes the destination's location relative to your current location.
Speaker A
By default, ls prints the names column of the table. For example, here we see two names representing the fact that our current directory contains two items.
Speaker A
Paths beginning at the root directory, by contrast, are called absolute paths. This often gets taught as though there are two different types of paths with fundamentally different behavior.
Speaker A
You recall that a single dot represents the current directory, while two dots represents the directory above.
Speaker A
And while there are obviously different semantics involved with absolute and relative paths, it's the same fundamental routing operation.
Speaker A
So, ls hides them. We can ask it to stop by using ls minus a, where a stands for all.
Speaker A
The only difference is where the path begins, and that's decided only by whether the first character is a slash.
Speaker A
This fact means that we can create our own hidden files by giving them a name that starts with a dot.
Speaker A
The phrase "wherever you currently are" is doing a lot of work in defining relative paths, and it's fairly meaningless without further clarification.
Speaker A
A hidden file isn't special in any way. The file system doesn't know or care that it's hidden.
Speaker A
Like, what does it even mean to be somewhere in the file system? I'll have to come back to that question later because it's a cleaner explanation with a few more pieces in place.
Speaker A
Now, this table that we call a directory has to be stored somewhere. And on a Unix system, the answer to where is this data stored is most always the same.
Speaker A
For now, let's stay with the walk itself. We said that it goes one name at a time from left to right, but what are we walking through exactly?
Speaker A
And if a directory is a file, then we should be able to read it like a file.
Speaker A
It looks obvious enough with a pretty animated tree on screen, but there's no tree built into the computer.
Speaker A
We can use cat to examine the contents of our notes file, for example. cat notes.text prints out the contents of notes.text.
Speaker A
The disk is just a wasteland of bytes. There's no names, no folders. It's just an enormous expanse of number blocks.
Speaker A
We're not told that there's no such file. We're told that it's a directory. So, the system isn't confused about what's there. It's just drawn a distinction that we've not yet accounted for. In other words, somewhere the system has recorded that notes.text is
Speaker A
And somewhere between that and /bin/csh, there sits a system that can turn an arbitrary human-readable name into the right bytes every time.
Speaker A
ls supports a minus l flag where the l stands for long. It prints out more information about each entry in the table or a long listing if you will.
Speaker A
To me, this is such an amazing feat of ingenuity, and it's so often taken for granted.
Speaker A
There's a good amount of information here and each piece has its own story. For now, if we look at the first character of each line, every regular file gets a dash while both directories get a d.
Speaker A
So, before I show you how it's done, have a think about how you might do it yourself.
Speaker A
Now we have a model we can work with. A directory is a file whose contents are a table and each row in that table maps a name to a number.
Speaker A
The simplest idea is one enormous table with every full path on the left and the location of its bytes on the right.
Speaker A
We'll see how exactly in just a minute. Before we chase that number though, I want to stay on the name side for just a little longer because a few practical notes about file naming fall right out of this model.
Speaker A
That could work, and it'd be fast, too. But, think about what happens under this model if you rename just a single folder.
Speaker A
A slash because that's the separator we use for path resolution and the zero byte because that's how the system knows where a path ends.
Speaker A
Every single file underneath it, every project, every dependency, potentially hundreds of thousands of rows, would have to be found and rewritten because each one recorded its full path.
Speaker A
File extensions work the same way. The dot in notes.text is just another character in the file's name and dot text means nothing to the system at all.
Speaker A
This means that the table would also store lots of redundant information, which is less than optimal.
Speaker A
All right, let's finally go ahead and see what this number is all about and how the system keeps additional information about each file.
Speaker A
So, what if, instead, we extend the tree all the way down to the disk itself?
Speaker A
An inode is a record that holds everything the system knows about a given file, including its size, its type, its owner, when it was last modified, and perhaps most importantly, pointers to the actual file's content on disk.
Speaker A
We'll give the whole disk to the root directory and let each nested directory subdivide its parent's region.
Speaker A
The exact shape of the inode store is decided by the file system implementation. What's consistent is that whatever the structure is, it's indexable by number, and that index is the inode number.
Speaker A
This looks tidy, but it's horribly impractical. You'd have to decide how big each region is before knowing what's going into it.
Speaker A
We can use stat with a file's name to peer into the corresponding inode. Here, we can see the file size, its type, its owner, and when it was last modified.
Speaker A
And if you get it wrong, you're back to shifting everything around and making hundreds of thousands of edits for a simple operation.
Speaker A
What's not part of an inode is a file's name. You'll see one printed in stat's output, but that's just stat handing you back the argument you gave it. It's not coming from the inode.
Speaker A
What's more is that moving a file between directories would become a physical operation, since we'd have to copy every byte of it across the disk.
Speaker A
It's just a row in some directory that happens to reference it. What's clear now is that there are really multiple levels hidden behind the word file.
Speaker A
We can do better. Neither of these models is wrong about everything. The tree is the obvious conceptual model. It just can't be carved onto the disk directly.
Speaker A
And there's the file data itself, which is pointed to by the inode. From here, we're going to start pulling the structure apart and using the fact that the file's name, its metadata, and its content are all separable to do
Speaker A
And looking things up in a table is a good instinct. The problem is that it stored full paths, so every row knew more than it needed to.
Speaker A
We started with a file path and examined it literally as a string. We found that rather than being a static address, it's actually a set of instructions walked one name at a time from left to right. Hence the term path.
Speaker A
The way we reconcile these two models is pretty clever. Instead of one enormous table for the whole disk, we'll give each directory a small table of its own with the name on the left and a number on the right
Speaker A
Then we investigated what exactly we were walking through and found that we were walking through a field of little tables called directories that map file names to numbers.
Speaker A
and allow those tables to refer to one another in a tree-like structure. This way, the root's table knows about bin and bin's table knows about zsh.
Speaker A
The reason that directories are able to store both files and other directories in the same table is that directories are themselves a special type of file.
Speaker A
Nothing anywhere records a full path because nothing has to. Each directory knows only of its own contents.
Speaker A
This inode structure is what's pointed to by the number in the directory. And it's also what points to the location on disk where the file's contents are.
Speaker A
If we rename a folder now, exactly one row changes in exactly one table. Everything underneath comes along for free because nothing underneath ever knew where it was relative to the stuff above in the first place.
Speaker A
Similarly, the inode points at files' bytes, but those bytes have no idea what's pointing to them.
Speaker A
And this is why the walk has to happen one step at a time. There's no table anywhere that can resolve /bin/zsh in one shot because that information doesn't exist in one place.
Speaker A
So, let's see how we can exploit the properties of these data structures to do things that might otherwise seem counterintuitive.
Speaker A
And it's worth clarifying a directory doesn't just have a table, it is a table.
Speaker A
But nothing about a directory structure requires that. A directory is a table. What's stopping me from writing another row to the table with a different name, but reusing the same number?
Speaker A
Let's go ahead and take a look at one. The c
Speaker A
We can link two files like so. First, we'll create a file called A. The arrow here is called the redirection operator, and it sends echo's output into a file called A rather than to the screen.
Speaker A
We'll learn more about redirection when we cover process communication. For now, we have a file called A that contains the word hello.
Speaker A
Now, we can link A to B using ln. ln AB writes a new row to the current directory's table with B as the name and A's inode number.
Speaker A
We can confirm this with ls -i. Given that A and B have the same inode number, you should be able to predict what's inside of B now.
Speaker A
It's the same as what's inside of A. A and B aren't two distinct files that happen to have the same content. There's literally one file with two names.
Speaker A
Neither one is the original and neither one's a copy. In a sense, this means that a file can in fact live in two places at once because two distinct names can both reference the same file.
Speaker A
You may wonder if these two names really reference the same file, then what happens if I remove one of them?
Speaker A
Does the other one simply cease to exist or does its name stop pointing at anything valid?
Speaker A
To find out, I'll use RM to remove A. We can check if B still exists by using LS.
Speaker A
And we can examine its contents by using cat. As you can see, B remains entirely intact even though we just removed what was supposedly the same file.
Speaker A
What this demonstrates is that RM is actually a bit of a misnomer. The operation underneath RM isn't remove, rather it's called unlink, which is the exact opposite of link.
Speaker A
One of the fields in an inode structure is called the link count and it tracks the number of names that point to the inode.
Speaker A
As you can see, for B, the link count is currently one. If we go ahead and relink B to A, then we'll see the link count for both files is now two.
Speaker A
It's only when an inode's link count reaches zero that its bytes are actually freed.
Speaker A
This is why B still exists even after I removed A, even though A and B were truly the same file.
Speaker A
There's a question that I left open at the start. I said that a relative path begins from wherever I currently am, but I didn't explain what that meant or where the system keeps it.
Speaker A
And where I currently am is a bit of a nonsense phrase in the context of a file system.
Speaker A
Thankfully, I've not been uploaded to the file system at all. I'm sitting here in the real world in front of a microphone.
Speaker A
What's actually meant by where I currently am is a working directory. A working directory is the directory that a running process treats as its sort of home base.
Speaker A
We can find the shell's working directory by using a command called PWD, which is short for print working directory.
Speaker A
This directory is where I currently am while I'm using the shell. You might assume that the working directory is stored as an environment variable, and you'd be right, sort of.
Speaker A
There is an environment variable called PWD that keeps the working directory, but this is just a convenience wrapper that the shell provides.
Speaker A
The source of truth actually lives in the kernel. The kernel keeps a working directory for every process, right alongside things like the process ID.
Speaker A
A process's working directory is inherited from its parent at birth, and changing it in the child does nothing whatsoever to the parent.
Speaker A
That last part may sound like a technicality, but it predicts something interesting. Think about CD, which is the command that we use to change our working directory.
Speaker A
Except, we just said that a working directory belongs to a process, and that a child process can't change its parents.
Speaker A
Which means that CD cannot effectively run as a child process of the shell. But all programs spawned from the shell run as child processes. So, what gives?
Speaker A
Let's see where CD is stored. It turns out that CD isn't a separate program at all, because it can't be.
Speaker A
If it were, the shell would spawn a child process to run it, and that child would dutifully change its own working directory, and then it would exit, taking the change along with it.
Speaker A
Hence why CD must be built into the shell. The distinction between the kernel's working directory state and the shell's self-reported working directory is usually invisible, but not always.
Speaker A
Check out what happens if I go into this directory called temp, and then print the working directory.
Speaker A
We get exactly what we'd expect. But, PWD takes a minus P flag. If we run PWD minus P, we get something different.
Speaker A
The minus P stands for physical, which rather implies that {slash} temp is something other than physical.
Speaker A
To see what's going on, let's inspect {slash} temp with LS minus LD. Here, minus L gives us the long listing, and the minus D says, "Tell me about this directory itself, rather than about the stuff inside of it."
Speaker A
You recall that directories typically keep a D to represent their file type, but this one has an L. That L stands for link, and it represents the fact that this directory is actually a symbolic link, or a symlink.
Speaker A
A symlink is a real file with its own I node. What makes it special is its contents.
Speaker A
It literally contains a path stored as raw text. When the walk reaches a symlink file, it reads that text, and substitutes it, and then carries on walking from there.
Speaker A
To create a symlink, we can use LN minus S, where S stands for symbolic.
Speaker A
For example, we can symlink Z shell to a file called Z in our current directory, like so.
Speaker A
When we examine Z, we can see that its file type is a symlink, and we can see that it points to Z shell.
Speaker A
Even more interesting is the size of this file. It's eight bytes, which is the exact same number of bytes as in the path {slash} bin {slash} zsh.
Speaker A
In other words, the file's entire content is the path that it's linked to. Since a symlink's target is represented as text, and the walk unconditionally picks it up. You can probably guess what happens if I point two symlinks at one
Speaker A
another. Nothing broke yet, because these two files are just sitting here. The problem occurs when I try to read one of them.
Speaker A
The system is unable to resolve the symlink due to the circular reference. It first attempts to read X, and then it's redirected to Y, then it's redirected back to X, and so on and so forth until eventually it's had enough
Speaker A
and calls it quits. Compare all this to what ln did without the minus s.
Speaker A
A hard link was literally another row in the table. A second name pointing straight at the same inode.
Speaker A
A symlink is another file entirely containing a path which has to be resolved all over again.
Speaker A
So, what does it mean for a file to live somewhere? The answer, really, is that a file doesn't live in one place.
Speaker A
Its name is stored in an inode, its metadata is stored in an inode, and its content is stored on the disk.
Speaker A
And as we'll see later, it's not even necessarily a contiguous block on the disk.
Speaker A
As is usual, [music] there's plenty that I left open here. Unfortunately, all of this will have to wait for a future video.
Speaker A
In the meantime, if there's something else that you'd like to see covered, please [music] do leave it in the comments, because that genuinely does steer what I make next.
Speaker A
And if this one was useful or entertaining, a subscription helps quite a bit. With that, I do hope you've enjoyed, and please take care.
Topics:filesystemfile pathinodedirectorypath resolutionabsolute pathrelative pathsymbolic linkshellfile system structure
