Background:

I'm mostly a vi user and am converting to Emacs.

Past the basics and now use it for writing code in C and some Python with plans to do an R conversion too.

I wrote (Common) LISP for years over two decades ago when I was in grad school but now I've severely lapsed (can write hello world but that's it).

Question:

What would you recommend for a guy in my position to get learning the Emacs LISP environment and start hacking Emacs.

Are there books, blogs, resources I should read, exercises I should do?

What a great question! Here is the path I've taken:

Step 0: Read the Emacs Tutorial

Read the Emacs Tutorial before anything else. It sounds like you've done this. Good on ya! However, for anyone coming to this at a later time, this is the place to start your Emacs journey. There are a lot of misleading blog posts out there. They over complicate things. Emacs makes it easy:

1. Start Emacs


2. Press <RET>
   

You can also access it via Ctrl + h t, or C-h t in Emacs notation.

The tutorial introduces the basic concepts of Emacs, clarifies historical terminology, and gets you going in under 30 minutes. It also sets the stage for the next step.

Step 1: Read "An Introduction to Emacs Programming"

The next step is to read An Introduction to Emacs Programming. I cannot recommend this book enough; it's so well written! It comes with most Emacs installs¹. Simply press C-h i and select it from the menu. You could also go to it directly via C-h i m Emacs Lisp Intro.

The Preface states,

This text is written as an elementary introduction for people who are
not programmers.

It should say "for people who are non-Lisp programmers." Lisp is a little different from other languages, so it's worth reading if you're unfamiliar with Lisp. The author has an approachable style and walks through many examples. It took me about a week to work through.

Step 2: Read OPC (Other People's Code)

2a. Read the source code

Because Emacs is "the extensible, customizable, self-documenting, real-time display editor", all the source code is included. Learn to use the help features, C-h f for a function and C-h v for a variable. There is always a link to the source code:

2b. Read all the init.el's

A lot of people track their init.el files in Git or Mercurial. Seek them out. One trick is to click on people's profile on StackOverflow, especially emacs.stackexchange.com:

They often have links to their GitHub or personal website:

Repos called dotemacs or dotfiles are quite common. Over time you'll get a sense of what good code looks like and what bad code looks like. You'll also learn the big names, like those of Xah, Sasha Chua, binchen, Magnar Sveen, abo-abo, and many more. There are so many people to learn from!

Step 3: Hack on Emacs

Challenge yourself to write the functionality you want. Never copy code blindly; always take the time to understand it. If you don't understand it, don't use it! When you take the time to understand a piece of code, aside from learning something new, you often think of ways to improve it for your particular use case. This is an opportunity to hack! After all, that's what your init.el is for! Make mistakes, break things, learn to debug. You'll find that because Emacs is written in Elisp, development is a delight.

Step 4: Ask questions on emacs.stackexchange.com

I've found that the Emacs community on StackExchange loves a good Elisp question. So, hack with confidence that other people will appreciate your efforts to learn.

Step 5: Answer questions on emacs.stackexchange.com

Nothing clarifies your understanding like trying to explain something. Trolling the main page, you'll also find answers to questions you never knew you had.

Conclusion

Over time you'll find yourself able to generate solutions faster and to answer other people's questions. Like all programming, it can be frustrating. However, remember to judge yourself today against yourself yesterday. Some people have been hacking on Emacs for, literally, 30 years! Most of the time, though, you'll find that Emacs is a pleasure to work with and the freedom it gives you is inspiring.

Happy hacking! :)

PS: Check out the package evil.

¹ Some installs don't come with it. Debian users, for instance, must install it separately. It's kind of silly, but it's stored in the non-free repos. Add a non-free repo to /etc/apt/sources.list by appending non-free to the end of each entry:

deb http://http.us.debian.org/debian stable main contrib non-free

I recommend the "Emacs Lisp Intro".

The "Emacs Lisp Intro" might be already available in info format within your Emacs. Try C-h i m Emacs Lisp Intro RET or evaluate (info "(eintr) Top") in Emacs.

If this fails search the web for "Emacs Lisp Intro".

1. 
   

   Start with the Emacs Wiki page Learn Emacs Lisp.

   
   
   

   Not that that page itself will teach you Emacs Lisp. It will instead point to learning resources -- exactly what you're looking for here, with your question. Many users have contributed to it and edited it over a period of years. That presents advantages as well as disadvantages wrt one user's blog. The quality of suggestions there is of course various, because of the variety of inputs.

   
   


2. 
   

   Beyond that - everything that others here have said, especially starting with the built-in doc of Emacs:

   
   
   

   • The Elisp manual - mostly a reference, but some good usage guidance and conceptual info too. It's built into Emacs: C-h i then choose the Elisp manual.

   
   

   • The Emacs Lisp Intro - step-by-step intro to Lisp and Emacs Lisp. Also built into Emacs and available from C-h i.

   
   


3. 
   

   Since you've used Common Lisp it will be super easy to pick up Emacs Lisp. Consider yourself lucky about this.

   
   
   

   If you had used Common Lisp (or Zeta Lisp or whatever) on a Lisp machine then the experience would be quite similar. You can think of Emacs as (among other things), a great Lisp environment. You can go from zero to 60 in no time, when learning.

   
   



4. Have fun!

Since you have a background in Common Lisp and you're "past the basics", there is no point getting lost in the "Introduction to Programming in Emacs Lisp".

What you can do right away (I don't have a background in CL and it worked for me) is:

1) find a mode that you use and try to add functions, you'll find help in the Emacs Lisp Reference

2) write your own custom file with your own simple functions, here again use the Emacs Lisp Reference

The help-gnu-emacs list is extremely helpful.

What would you recommend for a guy in my position to get learning the
Emacs LISP environment and start hacking Emacs.

I really don't think you need to read something specific on emacs lisp. It's no secret that if you want to do something in Emacs, chances are, you just have to find the right command or someone has written the code already and it now resides somewhere on a wiki page, or github, or was made into a plugin. Just by copy-pasting that code and reading it, you will have learned a fair share of Emacs lisp by the time you are even somewhat satisfied with your .emacs.

If you want to get some ideas on how to customize Emacs, custom configurations by other people might be a good source of wisdom. A selected list of such configurations you can find here. Though, really, having your own vision works better and visiting other people's .emacs is usually warranted only when you want some questions answered [e.g. how do I structure my code/packages?] or some code copied for some specific goal of yours.

Browsing MELPA got me to install and try maybe a dozen of packages.

If you are going to develop a plugin, I found that reading the code of some particular plugin of your liking can get you started fairly quickly.

Since you use vi, check out evil-mode, it has worked great for me. I would also recommend helm with helm-apropos (or just apropos if you don't want helm) command to make your journey easier.

So, my advice is to get to the customization point right away and you will learn everything you need on the go. Especially that you hacked lisp before, long ago or not, and because learning elisp is, as far as I can tell, a means to an end for you, which is to customize emacs.

As a refresher on Lisp I highly recommend the first three chapters of Peter Norvigs Paradigms of Artificial Intelligence Programming. Those give you a great and fast pace (re)introduction to Lisp which will get you back on track quickly.

For total noobs there is COMMON LISP: A Gentle Introduction to Symbolic Computation which teaches you the same concepts but at much lower pace.

Both books are available online, for free.

After going through one of the above, you can read all Elisp tagged posts on nullprogram.com and go through the reading guide at masteringemacs.org. To learn about common pitfalls reading the top questions here will be of help.

After doing the above the Info manuals will usually answer most remaining questions that come up when hacking on Emacs.

Seems to me that Loren Ipsum's answer covers it rather well.

https://emacs.stackexchange.com/a/47320/21433

I would simply add that I'm assuming you know the basics of Lisp. That would help you, but it is not necessary (I'll get back on that.)

One way to learn is to simply hack or modify an existing mode/extension. I learned it back in 93 by hacking at the Ada language mode file. It was one of the most fun experiences I had back in college.

So you could do that with a language of your choosing, say, a Java mode, or a Lisp mode, or even an Elisp mode file.

Change it, add macros, etc. That's how I did it.