ScrapySharp has a Web Client can simulate a Web browser (automatically manages the referrer, cookies, etc …)

HTML parsing must be as natural as possible (close to the web dev), a CSS Selectors compatible with Linq has been implemented.

This framework encapsulates HtmlAgilityPack to add useful features.

The source code is here: code repo

It is very easy to use by installing Nuget.

Image of nuget

Here are some examples of basic use:

var divs = html.CssSelect(“div”);  //all div elements
var nodes = html.CssSelect(“div.content”); //all div elements with css class ‘content’
var nodes = html.CssSelect(“div.widget.monthlist”); //all div elements with the both css class
var nodes = html.CssSelect(“#postPaging”); //all HTML elements with the id postPaging
var nodes = html.CssSelect(“div#postPaging.testClass”); // all HTML elements with the id postPaging and css class testClass

var nodes = html.CssSelect(“div.content > p.para”); //p elements who are direct children of div elements with css class ‘content’

var nodes = html.CssSelect(“input[type=text].login”); // textbox with css class login

It’s also possible to match ancestors

var nodes = html.CssSelect(“p.para”).CssSelectAncestors(“div.content > div.widget”);