The problem was that the positive side of the battery didn't touch the terminal due to the short battery terminal and the construction of the keyboard. The simplest solution to fix this is to roll up a small piece of aluminum foil and toss it in there before the batteries.