In order to make icons wider in TinyMCE, you will need to remove the default width on the mceIcon class. Its forced at 20px, which limits all icons to be square.

To make the change, go to your ui.css file
For advanced theme users, its located in themes/advanced/skils/default/ui.css

In Version 3.2.2.1, its line 30:
.defaultSkin span.mceIcon, .defaultSkin img.mceIcon {display:block; width:20px; height:20px}

The line should be changed to:
.defaultSkin span.mceIcon, .defaultSkin img.mceIcon {display:block; height:20px}


Next, you will need to define your custom button when initializing TinyMCE.
Using the example:
setup : function(ed) {
// Add a custom button
ed.addButton('mycustombutton', {
title : 'My custom button',
image : 'img/[Filename of custom button]',
onclick : function() {
ed.selection.setContent('<strong>Hello world!</strong>');
}
});


Add the button to the menubar, which is in the section defined with “theme_advanced_buttons…”
For my example, I will need to append “mycustombutton,” to it

You will then need edit the ui.css file with the wider button. For simplicity, add it to the end of the ui.css file so you can easily copy it when a new version of Tiny MCE is released. Replace the boldface “mycustombutton” with the name of your button. If you button name is mrfancypants, then it will be .defaultSkin a.mce_mrfancypants

/* Custom button */
.defaultSkin a.mce_mycustombutton {
width: 65px;
}


That should be it! I plan on adding a lot more TinyMCE tutorials since I’ve decided to change the WYSIWYG component on all my applications to TinyMCE. CSS works really well with TinyMCE and cross platform browsers.