Using OpenLayers
Google Maps API is not the only option for developing a JavaScript mapping application. OpenLayers is one of the oldest and frequently used frameworks. It is an open source project constantly maintained and developed by a growing crowd of enthusiast developers. As you've noticed, it is used with the GeoServer previews.
Copying the OpenLayers previews don't do much good, so let's go over some basics with OpenLayers.