This was prompted by the fact that I noticed recently that there is not a lot of documentation available on the web regarding the F5 BIG-IP’s Web Scraping Protection mechanism and almost none regarding what it actually does to the underlying web page code presented to your end users.

Web scraping is defined as a computer software technique of extracting information from websites.  The people people running the web scraper program typically save the contents of what is scraped and use it for their own means.  Sometimes it is just for archiving purposes, such as Archive.org’s “WayBackMachine“.  Several companies even sell what is considered by many to be legitimate commercial web scraping software.  One such company is called Mozenda, who lists such clients as Microsoft, IBM and Citi.

But then there are the “Others” as I like to to call them.  This can range from hackers with bad intentions to companies simply seeking a competitive advantage over another company. One example of this that I  can think of dealt with a few websites who make their living by offering vacationing deals.  So these leaders of their industry would publish airfares for many popular destinations on their websites and their competitors would use a computer program to scrape the pricing off of their pages. They would then take this pricing, subtract a few dollars, load it into another program and update the pricing on their own website thereby making their vacation deal offerings just a little cheaper than their competitors!

Web scraping is not an illegal activity, but it can be against the “Terms of Use” for some websites.  Now, all of that being said, it is definitely nice to know that the BIG-IP ASM has a built in feature that you can enable to protect your own websites from being scraped.

It does this by attempting to determine whether a web client source is a human or if it is a headless computer program.  To do this it injects a piece of java script code into the headers of your HTTP traffic. I will not provide the full source code for the java script, but I will hopefully provide enough for those searching through Google to be able to find this page.

When you are viewing the web page being protected by an ASM and web scraping anomaly detection is being actively used to protect the web page you will see the following elements. To actually see these elements, open up Firefox, browse to the website in question and then right-click and select “View Source”. You should see a java script insert beginning very close to the top of the page that contains some of the following elements:

var jsepee
jsepee CSHUI_RANDOM_DATA_NODE
CSHUI_RANDOM_DATA_NODE’]!==undefined&&jsepee['
CSHUI_RANDOM_DATA_NODE
CSHUI_COOKIE_NAME']=jsepee['CSHUI_RANDOM_DATA_NODE
CSHUI_COOKIE_VALUE_TRUE']=’true’+'_’+jsepee
CSHUI_RANDOM_DATA_NODE
CSHUI_MONITOR_KEYBOARD’]=true;jsepee['CSHUI_MONITOR_MOUSE
CSHUI_MOUSEMOVE_EVENTS_TARGETCSHUI_MOUSEMOVE_LAST_X_LOCATION
CSHUI_MOUSEMOVE_LAST_Y_LOCATION']=0;
CSHUI_MOUSEMOVE_IS_CONTINUOUS
CSHUI_KEYBOARD_EVENTS_TARGET’]=1;jsepee
CSHUI_KEYBOARD_EVENTS_COUNTER

You can seen by looking at these events that it is looking for keyboard, mouse and other data to determine if the content is being looked at by a human or something that falls in the OTHER category. Once it has made a determination the web application security policy will follow whatever guidelines you have set under the policy settings.

So there you have it, yet one more reason why the F5 BIG-IP ASM is an excellent tool to be included in your defense in depth lineup.

Before I came to the class I could build a security policy and assign it to a website and do some minor tweaking.  Now I can say with confidence that I can build a web application security policy that is PCI compliant and has a solid foundation.

One of the main ingredients for a successful training session/class is you really need an excellent instructor.  If the instructor doesn’t know his stuff or doesn’t really enjoy the subject matter it can have a negative and direct impact on the course.  The class I took was lead by a gentlemen named Keith Bowers who has worked for f5 Networks for 10+ years.  Granted, I could be wrong about number of years, but I think I am close.  I can say for certain thought that Mr. Bowers knows the material and he seemed to really enjoy teaching the class.

This wasn’t the kind of class where you go and read along with the teacher word by word out of the book.  Keith gave very concise and well thought out lectures regarding each subject that we touched on.  I say concise because he said everything that he needed to in order for you to comprehend the material and to be able to apply in a real world situation.  Then he would provide guidelines for the hands-on portion of the lab for that section and turn us loose on the BIG-IP box that each student gets to all to his or her self.  When a student had trouble getting through a lab he would sit beside them, provide information on things to look for and provide clarification on things until the student got through the lab.  He was really good about teaching you to fish rather than just giving you an answer out of the teachers edition of the manual

So what kind of goodness can one expect to learn at an ASM 10.x course?  Here is a brief list of the things that we covered:

Installation
Web Application Concepts
Web Application Vulnerabilities (with instructions on how to perform a few basic hacks)
ASM Application Configuration
Security Policy Building
Creating Custom Attack Signatures
Reporting
Traffic Learning
Protecting XML and Web Services
And more…

On the second day that I was there I also had the chance to meet up with a few members of the DevCentral Core Team!  I was able to bounce out of class a little early so Joe met me outside the training room and proceeded to give me a tour of the place.  At one point I tried to slip a VIPRION into my cowboy hat and almost made off with it but the 30+ blue ethernet cables sticking out from underneath my hat gave me away.  Alas, I had to put it back.  <Sigh>  Seeing that I was upset though Colin, Jeff and Joe provided me sneak peak of their latest TOP SECRET project to get my spirits up.  After the tour that I was given, my spirits were definitely lifted!  I wish I could tell, I wish I could tell…. but I can’t.  It was awesome though.

We then proceeded down to Buckley’s Pub for some lunch and along the way we went over a little bit of history, talked about things that a tourist like me should do when visiting Seattle, etc…  Jeff kindly wrote up a blog article about it and even included a picture that he took of Colin, Joe and I at the pub.  You can check it out here:

http://devcentral.f5.com/weblogs/JeffB/archive/2010/04/01/1088132.aspx

I can’t provide all the details of what we talked about, I was having to good of a time to remember them all.  I know we talked about Bear Grylls (Man vs. Wild), Mac keyboard shortcuts and the MVP Summit… How those are all interconnected I will leave up to you to ponder… Hehehehe… seriously, thanks for a great time fellas.  And also thanks for what you do every day.

Well, if you have made it this far into my blog post you deserve a treat!  Below is a snippet of some videos that I took on April 1st during the training class, some footage from the TOP SECRET stuff they showed me and some footage from the pub!  I had to try out my f5 Networks MVP branded FlipMINO after all!  Sorry if it is a little choppy in a place or two, I had to compress it before I uploaded it to YouTube.

Camera In Cowboy Hat Video

After looking around on the F5 website and through a few reams of paper searching for that information, I discovered that it didn’t exist in the public domain.  Therefore I opened up a ticket with the kind folks over at F5 regarding the matter.  Below is the end result of their hard work:

bd – This process implements the ASM policy on the HTTP traffic it receives.
If not running: No Traffic Passes
Logs To: /ts/log/bd.log

bd_agent – Delivers policy configuration to the bd process and forwards bd events to the rest of the system.
If not running: No enforcer configuration updates, no statistics (not including request log)
Logs to: /var/log/asm, /ts/log/bd_agent.log

dcc – The DCC process forwards policy updates to the bd via the bd_agent, handles bd events received from the bd_agent, and manages communications with the rest of the system.
If not running: No enforcer configuration updates, no statistics (not including forensics)
Logs to: /var/log/asm, /ts/log/dcc.log

verify_dcc – A form of "watchdog" process that monitors the dcc process, and reports any failures to the recovery_mngr.pl, which handles restarting the dcc.
If not running: No monitoring of dcc availability
Logs to: /ts/log/verify_dcc.log, /var/log/asm

mysqld – The mysql database process holding the policy as well as logs and policy builder data.
If not running: Configuration will not load, no logging, no traffic passes.
Logs to: /var/lib/mysqld.err

verify_mysql – A form of "watchdog" process that monitors the mysqld server, restarts if it needed, and reports any failures to the recovery_mngr.pl process, which restarts the dcc processes, since they must reconnect to the DB after any failure.
If not running: No monitoring of mysql availability
Logs to: /var/log/verify_mysql.log, /var/log/asm

clean_db – Monitors ASM DB tables, and prevents them from exceeding pre-defined limits on table size.
If not running: No deletion of old database records, may fill the disk.
Logs to: /ts/log/clean_db.log, /var/log/asm

log_manager – In charge of ASM-specific log file tasks such as rotating and archiving the logs.
If not running: ASM debug logs (non syslog) will not get rotated to tar archives.
Logs to: /var/log/asm, /ts/log/log_manager.log

recovery_manager – The process is in charge of starting the ASM daemons in their proper order, restarting daemons when watchdogs report failures.
If not running: ASM will not recover from any failure.
Logs to: /var/log/asm, /ts/log/recovery_mngr.log

crawler_manager – Handles starting and stopping the policy builder via the GUI.
If not running: No control of PB actions.
Logs to: /ts/log/crawler_manager.log, /var/log/asm

learning_manager – Populates the learning tables that are used in the processing building policies.
If not running: No learning suggestions.
Logs to: /ts/log/learning_manager.log, /var/log/asm

attack_manager – Populates the "Attacks Reports", based on security events.
If not running: No statistics of attacks.
Logs to: /ts/log/attack_manager.log, /var/log/asm

nwd_core, nwd_ts, and nwd_dms – Multiple instances of "watchdog" processes that monitor ASM daemons, and attempt to restart them if they fail.  Reports failures to restart daemons to the recovery_mngr.pl process.  Covered in https://support.f5.com/kb/en-us/solutions/public/6000/500/sol6590.html
If not running: ASM daemons won’t get brought up on failure.
Logs to: /ts/log/nwd.log, /var/log/asm

OK, I will step down off my soap/sales pitch box now.  It is just hard to have any kind of a discussion in the realm of network security without mentioning the consequences of NOT implementing a solution to address those concerns.  In this post I am going to discuss some information on an implementation that I have been working on over the last few weeks.  Of course the normal disclaimers do apply. The solution I am going to discuss is just an example of one implementation.  There are a number of ways roll out one of these devices into your existing infrastructure.  And last but not least I recommend that you seek further guidance from a Certified F5 Product Consultant if you are planning on adding one these devices to your network.

Now for some background information about the network that this implementation will be carried out on. The client already has a BIG-IP 6400 unit set up and running in their live/production environment.  It is licensed for a wide variety of modules from F5 Networks including load balancing, SSL offload, WAM and GTM.  The client also has a BIG-IP 6400 setup as a failover peer to the production box just in case of a hardware failure.  However, the BIG-IP ASM 4100 unit that they have CANNOT be licensed for load balancing and is only licensed for a few modules, mainly the ASM and LTM modules.

Since the 4100 does not have a failover peer like the 6400 we had to come up with a way to ensure that if this device failed that it would not have a negative impact on network traffic. We also had to figure out a way to load balance traffic going through the device.

So how do you load balance traffic AND ensure that traffic always makes it to the target server when you only have one device and no load balancing module?  I have to admit I was stumped for a bit.  It was then that a friend (a wise man I must say) pointed out what is now obvious.  Use all of your available resources.  So the method of implementation that we chose addresses both of those issues by relying on the BIG-IP 6400 for Priority Group Activation and Load Balancing.  The solution is simple really:

On the BIG-IP ASM 4100

We created nodes each pointing to IP’s hosted on our production web servers. We created a separate pool for each node and then we created a Virtual Server (also known as a VIP) that references each pool.

On the BIG-IP 6400

We created individual nodes on the BIG-IP 6400 that point to the VIP’s that we created on the ASM 4100.  We then added those nodes to the existing pools for each web application. For each of those new pool members we set the Priority Group Activation to Less than 1 and assigned the new pool members to Priority Group #2. Therefore, if the ASM 4100 has a critical hardware failure all traffic will bypass the VIP’s associated to the ASM 4100 and will be directed to the other pool members that are available.

This solution has a wide variety of benefits including: preventing the client from having to load balance in two places, prevents the client from having to rewrite numerous complex iRules, eliminates the worry of having a single point of failured and all of that allowed the device to be rolled out in a very short amount of time.  One other important thing worth mentioning is that since the VIP to node address are a one to one mapping in this instance, it allows the security policy builder to ease into high traffic sites because you can apply the policy to one VIP at a time.  If you have four VIP’s all aimed at the same web application/service, using this method you can choose to build the policy based off just two of the VIP’s .  Which in a high traffic environment is less taxing on the policy builder and the ASM 4100 unit.

(ADDED 4-10-2009)

It has come to my attention that the stand alone version of the BIG-IP ASM 4100 cannot be used for load balancing period.  That feature has never been supported on the ASM 4100.

For those not familiar with the BIG-IP ASM™ 4100, it is an application firewall that has no equal.  It can be easily configured to protect your enterprise applications from a wide variety of attacks and help you meet key regulatory mandates like PCI DSS, HIPPA and SOX.  With the rapid rise of SQL Injection, buffer overflow and XML attacks, having one of these bad boys to protect your network is essential.

I have personally built numerous security policies and I used the deployment guide “Implementing a Security Policy for a Production Web Site or Application” as a guidance tool.  It can be downloaded off of the official F5® website at:

www.f5.com/pdf/deployment-guides/implementing-security-policy-dg.pdf

The deployment guide does an outstanding job of explaining how to setup a security policy and apply it to a production system without having a negative impact on traffic.  That’s a must in the environment that I work in and I am sure that holds true in yours as well.

Having created a few policies on my own I have learned a trick or two about the process and I will share those with you over my next few postings.  I will be covering how to customize your security policies, what exactly the processes associated with the ASM™ module do and go into detail about a load balancing trick that I learned from a wise man.  Below is a picture I took of the Web Applications screen after the initial configuration of a few policies while they were still being run in Transparent mode.  I should have those postings up over the next few days so stay tuned for more!