In my current work, my boss got a project about getting data from websites and posting into a system. Sadly, one of those websites lacks an API, so we needed to figure out a way to get information from there. The short answer is scraping. To make this story short, after evaluating multiple projects, I think the best option is to use CasperJS.
CasperJS is an open source navigation scripting testing utility written in Javascript for the PhantomJS WebKit headless browser and SlimerJS (Gecko). It eases the process of defining a full navigation scenario and provides useful high-level functions, methods syntactic sugar for doing common tasks such as:
To install CasperJS, I did the following.
We are done! To call the Jasper do a command like this:
PHANTOMJS_EXECUTABLE=/usr/bin/phantomjs19 ~/casperjs-1.1-beta3/bin/casperjs
blog comments powered by DisqusAbout
Read about IT, Migration, Business, Money, Marketing and other subjects.
Some subjects: FusionPBX, FreeSWITCH, Linux, Security, Canada, Cryptocurrency, Trading.