ScrapySharp has a Web Client can simulate a Web browser (automatically manages the referrer, cookies, etc …)

HTML parsing must be as natural as possible (close to the web dev), a CSS Selectors compatible with Linq has been implemented.

This framework encapsulates HtmlAgilityPack to add useful features.

The source code is here: code repo

It is very easy to use by installing Nuget.

Image of nuget

Here are some examples of basic use:

var divs = html.CssSelect(div);  //all div elements
var nodes = html.CssSelect(div.content); //all div elements with css class ‘content’
var nodes = html.CssSelect(div.widget.monthlist); //all div elements with the both css class
var nodes = html.CssSelect(“#postPaging); //all HTML elements with the id postPaging
var nodes = html.CssSelect(div#postPaging.testClass); // all HTML elements with the id postPaging and css class testClass

var nodes = html.CssSelect(div.content > p.para); //p elements who are direct children of div elements with css class ‘content’

var nodes = html.CssSelect(input[type=text].login); // textbox with css class login

It’s also possible to match ancestors

var nodes = html.CssSelect(p.para).CssSelectAncestors(div.content > div.widget);