To get at the batteries you open a little circular cover on the right hand side and slide the batteries out. Or not. If one of the batteries has split open (even a small amount) it becomes jammed inside the slot and there's so little room that it's very hard to get it out.
There's a battery in there. Poking something into the hole doesn't help because you end up jamming the battery in further. The only viable option is to hit the keyboard on a hard object (e.g. the table) repeatedly and violently until the battery comes out.
It was a painful experience taking a piece of Apple hardware and whacking it on the table.