To answer your question simply, both of the sites above are showing a static image. Then, when the user clicks on the image, it opens up a modal window to show the video.
The first site in your post uses a popular plugin called Lightbox: http://www.huddletogether.com/projects/lightbox2/
The second site in your post uses a plugin Pretty Photo: http://www.no-margin-for-errors.com/...ightbox-clone/
I have used LightBox before but I have not used Pretty Photo. Basically, these plugins allow you to create modals that gray out the background of your site and open your image, video or Flash file in a window. The user is then immersed in your content and everything else grays out.
Lightbox is fairly simple to implement if you have some experience with HTML, but it only supports images / slideshows. There are modifications to the script, however, that will make it work with videos. Just do some Googling and you will find it.
Hope that helps.