I struggled with this one. First I though the problems were rooted in the MySQL version I was running (5.1.X). I realized finally that I had not provided all the configuration parameters.

My original attempt:

sudo gem install mysql -- --with-mysql-dir=/usr/local/mysql

The output from this command includes:

Building native extensions.  This could take a while...
ERROR:  Error installing mysql:
    ERROR: Failed to build gem native extension.

/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby extconf.rb install mysql -- --with-mysql-dir=/usr/local/mysql
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lm... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lz... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lsocket... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lnsl... no
checking for mysql_query() in -lmysqlclient... no
*** extconf.rb failed ***
...

I managed to make it work by changing the command line to:

$ sudo gem install mysql -- \
--with-mysql-config=/usr/local/mysql/bin/mysql_config
Building native extensions.  This could take a while...
Successfully installed mysql-2.7
1 gem installed

PS: The “\” is just a line continuation marker, to make things fit in this post.

PS: This was after downgrading mysql to 5.0.X.