mjhjmtu

I'm doing a unique sort on a concatenated set of index files where the first column will sometimes change between each index and the second column will be a key value (actually hex addresses). Each indexN file iteration records addresses that changed since the prior one -- if address 0xaa11 exists in index3, in the merged+sorted output it should replace the 0xaa11 address references from index1 and index2.

The question is, can I do this type of merge reliably with a tool like GNU sort if I merely pipe each source index to sort -u in a certain order?

For example, pipe indexes newest to oldest:

cat index3 index2 index1 | sort -u -k 2,2

When I test this, it does seem to preserve the lines from index3 containing addresses that also appear in index2 and index1, while removing those duplicate references coming from index2 and index1.

But will that always be the case? The sort man page is vague about this:

-u --unique output only the first of an equal run

I don't know enough about GNU sort's algorithms to predict whether lines with matching keys will always sort into the same order in which their source files were concatenated (e.g. the order they appear in the source stream). But I do know that sort algorithms don't always work in a linear fashion. That's why I'm looking for clarification of what sort's documentation seems to imply.

sort does not guarantee the order of lines that are equal for its purposes, unless you explicitly request this with the -s switch (--stable: stabilize sort by disabling last-resort comparison) – a stable sort algorithm is one that does not change the order of equal items.

However, the info page informs us that -u "also disables the default last-resort comparison", so yeah, you should be fine, but it's entirely not obvious from the manpage.