In addition to PubKey recovery here are just a few other "quirks":
* Transaction Malleability (Bitcoin Specific). It is cumbersome to fix to fix today but with a different txn structure it would be impossible.
* Signature Malleability (ECDSA). ECDSA can have multiple signatures for the same digest the solution is to either not make the signature part of the txn hash or limit Bitcoin to a single form.
* DER encoding serves no purpose. Even if OpenSSL was used the DER bits could be striped.
* Lack of Compressed Keys. Optimally the only valid key would be a compressed key and thus not only is key size reduced but there is only one format to consider.
* Unusual choice of double hashing. Normally done to prevent length extension attacks which don't exist in Bitcoin.
This isn't to say Satoshi wasn't a genius, the magic in Bitcoin isn't the cryptographic primitives used. It is in the way he elegantly used existing systems (digital signatures & hashing algorithms) to create a timestamp and consensus finding system that is very simple and yet very difficult to attack.