Why this file allocation test looks broken: st_size changes, but the disk blocks do not

The issue is usually not a bad allocator at all. It is a mismatch between logical file size, allocated blocks, and what your filesystem actually guarantees for calls like ftruncate(), lseek(), write(), or platform-specific preallocation APIs. A file can appear to grow immediately in metadata while remaining a sparse file with little or no physical space reserved.

Understanding the Root Cause

This bug report typically comes from a C test that checks file size before and after some form of allocation, then assumes the filesystem must have physically reserved that space. That assumption is often wrong.

On POSIX systems, several operations affect a file differently:

• ftruncate(fd, size) changes the file’s logical length. It does not guarantee full physical allocation.
• lseek() followed by a small write() can create a hole, which makes the file sparse.
• posix_fallocate(fd, offset, len) is the API intended to reserve space and avoid later allocation failures.
• fallocate() on Linux can preallocate space, but behavior depends on flags and filesystem support.

The core reason the test behaves unexpectedly is that the code likely uses an operation that updates st_size but then validates allocation using assumptions about st_blocks or later writes. Filesystems such as ext4, XFS, Btrfs, APFS, overlay filesystems, and network-backed filesystems may optimize allocation lazily. That means:

• The file size can increase immediately.
• The actual disk blocks may be allocated later.
• The block count may vary by filesystem, kernel, mount options, or copy-on-write behavior.

If the issue is inside a library or runtime that claims to “allocate” a file, the bug is often that it uses truncate semantics instead of a true preallocation call. In other words, the implementation creates the right length, but not the promised storage guarantee.

Another common cause is checking the wrong metric. st_size reports visible file length. st_blocks reports blocks actually allocated, often in 512-byte units. They are not interchangeable.

Step-by-Step Solution

The fix is to use a real preallocation API when the code needs guaranteed backing storage, then verify both file size and allocation behavior correctly.

1. Inspect what the current code is doing

If your code currently does something like this, it only changes the logical size:

int fd = open("test.dat", O_RDWR | O_CREAT, 0644);
if (fd < 0) {
    perror("open");
    exit(1);
}

if (ftruncate(fd, 1024 * 1024) != 0) {
    perror("ftruncate");
    close(fd);
    exit(1);
}

This may create a 1 MB file that is still mostly or fully sparse.

2. Replace logical growth with real preallocation

For portable POSIX behavior, prefer posix_fallocate() where available:

#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(void) {
    int fd = open("test.dat", O_RDWR | O_CREAT, 0644);
    if (fd < 0) {
        perror("open");
        return 1;
    }

    off_t size = 1024 * 1024;
    int rc = posix_fallocate(fd, 0, size);
    if (rc != 0) {
        fprintf(stderr, "posix_fallocate failed: %s\n", strerror(rc));
        close(fd);
        return 1;
    }

    close(fd);
    return 0;
}

This is the safest fix when the bug is that the program claims to allocate storage but only changes file length.

3. On Linux, use fallocate() if appropriate

If your target is Linux and filesystem support is known, this may also work:

#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(void) {
    int fd = open("test.dat", O_RDWR | O_CREAT, 0644);
    if (fd < 0) {
        perror("open");
        return 1;
    }

    off_t size = 1024 * 1024;
    if (fallocate(fd, 0, 0, size) != 0) {
        perror("fallocate");
        close(fd);
        return 1;
    }

    close(fd);
    return 0;
}

Be aware that fallocate() is not equally supported everywhere, and some filesystems emulate or reject it.

4. Validate the right properties

Use fstat() to check both logical size and allocated blocks:

#include <sys/stat.h>

void print_file_info(int fd) {
    struct stat st;
    if (fstat(fd, &st) != 0) {
        perror("fstat");
        return;
    }

    printf("st_size   = %lld\n", (long long)st.st_size);
    printf("st_blocks = %lld\n", (long long)st.st_blocks);
    printf("st_blksize= %ld\n", (long)st.st_blksize);
}

If your original test only prints file size, it may falsely conclude allocation succeeded. If it only checks block count, it may fail across filesystems with different allocation strategies. A robust test should state exactly what behavior is expected.

5. Update the test to match the intended contract

If the contract is:

• Create a sparse file — then ftruncate() is valid.
• Reserve disk space up front — then use posix_fallocate() or fallocate().
• Ensure future writes cannot fail for lack of space — then sparse growth is not enough.

A corrected test might look like this:

#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/stat.h>

static void print_file_info(int fd) {
    struct stat st;
    if (fstat(fd, &st) != 0) {
        perror("fstat");
        return;
    }

    printf("size=%lld blocks=%lld blksize=%ld\n",
           (long long)st.st_size,
           (long long)st.st_blocks,
           (long)st.st_blksize);
}

int main(void) {
    int fd = open("test.dat", O_RDWR | O_CREAT | O_TRUNC, 0644);
    if (fd < 0) {
        perror("open");
        return 1;
    }

    print_file_info(fd);

    int rc = posix_fallocate(fd, 0, 1024 * 1024);
    if (rc != 0) {
        fprintf(stderr, "posix_fallocate: %s\n", strerror(rc));
        close(fd);
        return 1;
    }

    print_file_info(fd);

    close(fd);
    return 0;
}

This version tests actual preallocation instead of mere size extension.

6. If this is a library bug, fix the implementation wording too

If a function is documented as “allocating” a file but only calls truncate, either:

• change the implementation to use posix_fallocate(), or
• change the documentation to say it creates a sparse file of the requested size.

That distinction matters because users rely on allocation guarantees to prevent write-time failures.

Common Edge Cases

• Filesystem does not support preallocation fully: Some filesystems may return errors like EOPNOTSUPP or behave differently with copy-on-write storage.
• Network filesystems: NFS and similar backends may not reflect allocation exactly like local filesystems.
• Container or overlay environments: OverlayFS and thin-provisioned storage can make allocation appear inconsistent.
• Copy-on-write filesystems: On Btrfs or APFS, preallocation semantics may differ from traditional extents, especially under snapshots or reflinks.
• Disk quotas: A sparse file may be created successfully, but later writes can still fail if quota or free space is exhausted.
• Wrong metric in assertions: Comparing only st_size or expecting an exact st_blocks value can make tests flaky across systems.
• Alignment assumptions: Allocated blocks may round up based on filesystem block size, so exact block counts are not always portable.
• Large file support: If the build environment lacks proper large-file flags, tests involving big offsets may produce misleading results.

FAQ

Why does ftruncate() make the file bigger without using disk space?

Because it changes the logical end-of-file. Most filesystems can represent unwritten regions as holes, creating a sparse file instead of allocating every block immediately.

Should I test st_size or st_blocks?

Test st_size if you only care about visible file length. Test allocation behavior with st_blocks or by validating successful preallocation calls if you care about reserved storage. Do not treat them as equivalent.

What is the safest API for guaranteeing space up front?

posix_fallocate() is generally the best high-level choice when available. On Linux, fallocate() can also be used, but support and exact behavior depend on the filesystem.

The short version: this is usually not a mysterious file allocation bug. It is a semantic mismatch. If the code needs real disk reservation, use a true preallocation API. If it only needs file length, then sparse behavior is expected and the test should not assume physical blocks were allocated.