You can style the widget containers at ATO->Widgets->Widget Container: - For a solid black line use: 'border-bottom: solid 1px #000000;' BUT remember styles applied here will effect all widgets.
If in addition, or in case you just want to effect one widget, you need to look at the generated source and find the '<div id=...' and using that , create a css insert (ATO->HTML/CSS Inserts->CSS Inserts
If you wanted to put a red box around the pages widget, looking at the generated code for a page we see the pages widget's 'DIV' is
HTML Code:
<div id="pages" class="widget widget_pages">
so the CSS insert would be
HTML Code:
#pages {
border-style:solid;
border-color:red;
}