mjhjmtu

Much like the VAX, the 6809 and 6309 have a preponderance of indirect addressing modes (in VAX parlance, "deferred"), though neither go so far as to include a VAX-like doubly-indirect mode.

To what extent were these modes useful/actually used?

I can obviously think of contrived instances where these would be a good fit, but it's difficult to determine whether these would show up in the "real world". It's worth noting that the 6809's cousin, the 68HC12, removed all but two of these indirect modes.

The 6809 offers to combine indirect with either a fix extended (16 Bit) address or all indirect indexed modes. These modes are extreme useful for all kinds of linked list and/or table processing. essentially the basic for any operating system, thus simplifying and speeding up OS programming including any kind of list defined I/O. Especially the later is quite powerful in a concurent setup (*1). And that's where the 6809 was targeted at: modular multi-process environments. Such an environment is filled with dynamic structures that are build and managed at runtime, usually not holding much predefined addresses and structures.

The 68HC12 was optimized as a sinclechip controller, not a general purpose CPU. Unlike a general purpose environment, a controller does usually operate on predefined structures with no or just a small amount of dynamic structures.

*1 - Think for example an I/O subsystem where each pending request is defined with a control block holding the address of a data buffer and a pointer within. During an interrupt to send or receive a character X could point to the control block for the actual operation (so no need to move that block into a fixed address) and use the actual buffer pointer within the structure with [X+offset] in a single operation. Increment (or decrement) it the same way and finish the I/O routine.