About:

These are just preview images of stock & etf data I've been mining for several months. Sorry, no styling, js, bells, or whistles here yet.
Just something I wrote up real quick to showcase what I'm developing.
Please note that this site and most of the content are very much incomplete and still in development.
The final product will have much more content beyond simple proprietary rating tracking.

Specs:

All of this runs on a single VPS: 4 GB RAM, 4 CPU cores, 96 GB storage, 40 Gbps In, 500 Mbps out. Memory & time efficiency are relatively good, but could be improved.

Back-end Scraping Infrastructure

~40 crawlers written in Perl with Moose legally and non-maliciously scraping ~20 websites utilizing PhantomJS, Tor, MySQL, memcached, and a rotating queue of thousands of active public proxy servers.
Will be looking into performance-focused NO-SQL databases for the data that doesn't require the overhead of a relational database model.

Back-end Web

Django 1.9 / Python (V2 will be done on Rails)

Front-end

Nothing special, just your typical JavaScript, jQuery, HTML5, and CSS with a couple plugins mixed in. CanvasJS used for the charts. Datatables used for the tables.
Will likely be adding in Angular and Bootstrap because they're all the rage these days. Also, they make stuff look cool.


Questions? Comments? Concerns? dan@streetscrape.com

Preview

All ratings are scaled 0 to 100 where 0=Strong Sell, 50=Hold, 100=Strong Buy

Zacks



Barchart



The Street



Aggregate View



Aggregate Table



Zacks Table