You can access the material library by selecting the material icon in the file browser.
The materials stored in this library are standard blender materials. There is a great deal of information available on material creation in Blender. Here are a couple of sites that can help you out.
Fluid Designer automatically generates the thumbnails for the materials all you have to do is save a blend file with the material you would like to save to your library into one of the categories in your library. You can determine the location on your hard drive by clicking the "Open Location in Browser"
Once you save a blend file into your library you can create the thumbnail by clicking the Create Material Thumbnails button. This will open an interface and show you all of the materials that were found in library. Now just check all of the materials you want to create the thumbnail for and click the OK button.
The render process may take several minutes, but once it is finished you will need to refresh the material by selecting the material category.