Update 2023: Due to updated to Twitter’s API you can no longer use the twitter widget to embed your timeline. This is due to changed made by Twitter (now X)

You can see the results of leaving a timeline widget  on the left side of this page.

You can still embed individual tweets by pasting in the url to a tweet.

This Widget is provided by the Jetpack Plugin.

To add it:

1. Activate the Jetpack Plugin
2. Activate Extra Sidebar Widgets in the Jetpack->settings section of the Dashboard.
3. Add the widget on the Widgets section of the Dashboard Appearance->Widgets.
4. You can add your twitter name to show your latest tweets.

More details on the jetpack blog:

Extra Sidebar Widgets – Twitter Timeline