>> ELF files have support for sections that are not loaded into the runtime
>> program image by the kernel.
>
> Correct.  The difficulties are social and not technical.  The tools you
> would need to extract such things conveniently, elftar(1) or mount-elf (1)
> and accompanying libraries don't exist and probably wouldn't be much used,
> thus would tend to be buggy.

I would explore other options before going for a tar archive or a file
system image. Those consist almost entirely of file metadata and
geometry/structural info that are not needed inside an ELF file.

All you really need is a (pathname data-offset data-size) tuple for each
file. Both random and ordered reads need to be fast; writes can be slow.

A prefix tree could be a good data structure for the index. Those
support fast random access and fast in-order traversal while retaining
good worst-case performance. They are also space-efficient since the
pathname prefix of each node is stored implicitly in its parent nodes,
not the full pathname in each leaf node.