Wednesday, July 2, 2008

Get link list from HTML page with regular expressions

Here's a little bit of code that I wrote to strip out the anchor links from an HTML page using regular expressions. The final line of the example code will only work if you're testing this in Joseph Albahari's excellent LINQPad. i.e. Dump() is an extension method that he's added to IEnumerable.

String text = @"<html>
                <head><title>Development Projects</title></head>
                <body>
                <ul>
                    <li><a href=""http://linttrap.domain.com"">linttrap</a></li>
                    <!--
                    <li><a href=""http://help.domain.com"">help</a></li>
                    -->
                    <li><a href=""http://help2.domain.com"">help2</a></li>
                    <li><a href=""http://help3.domain.com"">help3</a></li>
                    <li><a href=""http://gdhelp.domain.com"">gdhelp</a></li>
                    <li><a href=""http://helpadmin.domain.com"">help admin</a></li>
                </ul>
                </body>
            </html>";
Regex linkRegex = new Regex(" href=\"([^\"]*)\"");
List<String> links = new List<String>();
MatchCollection matches = linkRegex.Matches(text);
foreach (Match m in matches) {
    links.Add(m.Groups[1].Value);
}
links.Dump("Links");

Here's the output from the Dump() function:

▪ Links

5List<String>

http://linttrap.domain.com

http://help.domain.com
http://help2.domain.com
http://help3.domain.com
http://gdhelp.domain.com
http://helpadmin.domain.com

 

2 comments:

  1. What if we have links in the head tag? We don't need those.

    ReplyDelete
  2. thanks!!
    thanks!!
    it's example little and functional!

    ReplyDelete