Firefox's Web Developers Toolbar
The Libraries' Web Style Guide Committee is planning on putting together a Web Developer's Toolkit, which will include the style guide itself, color charts of suggested color combinations that match the site, and much more. One of the things I'm hoping gets included in the Toolkit is the Firefox Web Developers add-on. This is a tool I use every day for developing pages, and I hope that other library developers will give it a try. Besides Dreamweaver, it's the most important development tool I use.
I recommend downloading and playing with it to get a feel for its many, MANY features, but I'd also like to highlight a few that I use every day:
1) CSS drop-down (with real-time, in-browser CSS editing!)
The extension lets you view and manipulate a page's style sheet(s) without having to change the CSS on the server. I use this all the time because it's much safer to troubleshoot style sheets here than it is in Dreamweaver. You can view any style sheet the page is using, disable any or all of these style sheets or styles, and best of all, edit the styles.

Notice that in the below screenshot there are 2 CSS tabs on the bottom half of the screen. One shows the embedded styles, and the other shows the style sheet attached to the libraries home page. You can make changes to either of these style sheets within the browser, click on the home page (the top half of the screen), and your changes will be applied to the page (*note: the changes are only applied in the browser, not in the server. If you like the change and want to use it, you'll still need to edit the style sheet on the server.) It's very simple, and very useful for figuring out where problems on the page might be, or just playing with div positioning, font sizes, or colors.

2) HTML and CSS validation, including Section 508 validation.
Uses the W3C validators and the Cynthia Says 508 validator. This is very useful for troubleshooting browser-specific problems, and ensuring accessibility.
3) Page Zoom and resize (including 800x600 resize).
There are other ways to do these, of course, but it's nice to have them in one place.
4) Ruler toolbar
So you can measure anything on the page.
5) Outline block-level elements.
This outlines all the divs on a page, and can be used in conjunction with "Display ID & Class details" to see the names of all divs in use, as well as LOTS of other information.
6) Display image file sizes.
Actually, there are lots of things you can do with images using the toolbar-- probably more than you'd every need to do.
7) Disable javascript.
This is another one that you don't need a toolbar for, but it's faster than using the browser menu.
There's lots more available, including some things I just discovered while writing this posting. As one review puts it: "This is really the swiss army knife of web developer tools. It is so feature packed that I am still finding new things that it does. Unfortunately, the UI is also so cluttered that I am still finding new things that it does." (link)
The Web Developers extension may not help you improve your HTML or CSS skills, and it doesn't help you create pages, but it's great at locating and troubleshooting problems in existing pages. Highly recommended.
By the way, there's also an extension called Firebug that I haven't used, but that is supposed to really help create CSS and JavaScript.