LDOs drop the difference in voltage as heat, yes - hence the voltage difference between input and output determines the efficiency (eg a 3.0v LDO fed by 6.0v is 50% efficient).
In many cases where LDOs are used in designs the amount of lost power is negligible as the current being drawn is so small - simply not worth using a switcher for that rail. Also, LDOs typically have cleaner output so often an LDO is used to isolate an analog subsystem from noise on the main (digital) system rails.