Lantern is new crowdsourced project by developers of Limewire to thwart Internet censorship around the world.  The company describes how the technology works, “…running Lantern, every user with uncensored access can become an access point for those without, providing gateways to censored sites like Twitter, Facebook, and YouTube”.   Keep in mind, Lantern is not designed to preserve your anonymity and privacy.  Lantern will help you get a message outside areas where there is censorship.

The site provides additional information and a video you can watch.  There are also various ways to contribute to the crowdsourced project for those interested.


The most difficult thing about Raspberry Pi is determining which Raspberry Pi project is most interesting to you.  A while back I wrote a Java program to measure my Internet connection bandwidth.  I was thinking it might be a cool project the project over to the Raspberry Pi, write a JavaFX display, and display it like a picture frame on my desk.  Maybe build my own WIFI Pineapple security tool.  I’m still not sure what I want to do but first I need to get the hardware working.

A little background, at my home everything uses the Internet.  I have in the range of 40 to 50 Internet connected devices online at any one time, multiple managed switches, ISP gear, VOIP services, iPhones, solar monitoring, etc.  Do I really really need all this technology?  Nope, not at all but it’s just fun to play with.  That is, until I blow up my home network in the middle of my wife’s phone call or the kids Netflix isn’t working.  The family knows just where to go when the our home IT is not working so keeping my customers happy is important.

For my project I have a basic Raspberry Pi computer and 2.8″ TFT display.  I will follow-up with the nuts and bolts of the hardware list at the end of the article.  To begin all Raspberry Pi projects require an image to boot.  If your comfortable with *nix installs then you will feel comfortable setting up a Pi.  Download the Raspberrian image.  Write the image to an SD card.  On OSX, I used the ApplePi-Baker to write the image to an SD card.  My Mac Air just happens to have an SD card slot – whoot!

For me, getting a Pi working was not enough.  I wanted to the get the small 2.8 TFT display working with the Pi as well.  There are some subtleties of the TFT display, and perhaps other TFT displays, to be aware of.  In my case, the screen I’m using uses the GPIO pins so it may not be compatible with other hardware that may use the same pins.  Meaning if you are planning a bigger project there is some reason to to stick to the available composite video port or better yet the HDMI port.  The next concern is the display requires some modifications to the kernel.  All these details are included on AdaFruits web site.  Modifications aside, the results are impressive and the display only uses about 70ma.  Very low power.

After getting the computer and display together and basic testing out the way.  I thought I would try their sample movie and open source animation, “Big Buck Bunny”.   It ran pretty well but since my Pi does not have sound I’m missing half the story.  In any case, the animation framerates appear reasonable to the eye.  I notice the movie is optimized for the small display size.  I believe if your using the HDMI port there is hardware acceleration so it’s probably a bit more forgiving when playing different movies but I have not tried it yet myself.

Next, I got X-Windows and the desktop working.  Interestingly enough, the screen saver is set by default.  After a few minutes it turned off my screen and I thought the computer crashed.  A quick push to the touch sensitive screen and my computer was alive again – phew!  Of course, it’s difficult with my chubby fingers to push small UI widgets on the screen.  Any graphical application you write will have to be optimized to make use of the full screen width and keyboard shortcuts.  For the finishing touches I purchased a PIBOW 3d printed enclosure (on right) that accommodates the screen.  Everything works surpassingly well out of the box and following AdaFruits instructions.  It would be great if the PIBOW case included a smoked acrylic basel but mojo bang for the buck.  Enjoy!


My parts list is as follows.

Raspberry Pi Model B 512MB RAM
PiTFT Pibow kit
Adafruit PiTFT – 2.8″ Touchscreen Display for Raspberry Pi
32gb SD Memory Card
USB charging cable
Ethernet cable
*Core components purchased from AdaFruit.

You may have assumed when you installed Adblock Plus all annoying advertisements on web pages you visit would be blocked.  After all with a product name like “Adblock Plus”, it’s not too outrageous to assume the product should block ads – right?  Well your sort of right, some ads are blocked, and some are not.  Read the fine print.

I visited ABC News web site not long ago to read one of their stories.  I noticed when arriving on their site that a video at the top of the article began playing automatically.  I knew I had Adblock Plus installed so I was a little puzzled why the video began to play.  Usually, Adbock Plus blocks not only ads but any Flash embedded in a web page.  At first I thought, maybe the video was not Flash but another technology.  To check, I right clicked on the image and sure enough, the familiar Flash menu displayed.  The video was using Flash but how could that be possible?  I had Adblock Plus installed.  I noticed in the lower right hand corner of my Firefox web browser that the Adblock Plus icon was gray.  This means Adblock Plus is turned off.  I thought that was strange it was disabled since I distinctly remember I turning it on sometime back.  Unsure of what was happening, I turned blocking back on and hit Reload in my browser.  The page reloaded and when it did Adblock Plus turned itself off again and the video began to play.  At this point, I knew something was strange was happening or Adblock had a bug.  I figured I would visit the Adbock Plus web site to see if I could find any information on the subject.

It turns out Eyeo is the parent company that owns Adblock Plus.  Eyeo provides information about their blocking policies and positions on advertising to the public on their web site.  Apparently, Adblock Plus is not designed to block all content.  The company notes, “Adblock Plus exists to save its users from annoying ads”.  A rubric defining annoying or rather Acceptable ads is provided on their web site but it’s evidently not followed or enforced on some sites like ABC News.  According to the company, it’s possible to turn off all ad Blocking if it bothers you.

Unfortunately, unchecking does not work for me when I followed the instructions.  If I uncheck the box in the filters (as shown) and select “Disable everywhere” it blocks ads on other sites but when I return to ABC News and visit anywhere under the videos area advertisements continue playing on their own.  Perhaps this is a bug that slipped past the quality testing staff?

Adblock Configuration: uncheck to disable all ads

Incidentally, I don’t object to Internet advertising.  Without advertising, I’m not sure we would have any interesting content on the Internet to enjoy.  I also appreciate the pressure on Eyeo walking in the cross hairs of the Internet ad industry by blocking ads.  Perhaps there is a lesson, maybe we shouldn’t be dogmatic about ad blocking and take a more rational approach.  Taking Eyeo’s lead, perhaps industry can define an advertisement rubric and web browsers can include new features for users to adjust their ad preferences.  Eyeo’s approach considers acceptable ads from the user perspective in terms of different levels of distraction (e.g., moving images, etc).  As a security professional I look at ads differently and consider, risks of exploitation and privacy.  So for example, I would prefer to see ads as a simple JPG or image file – no JavaScript at all.  JavaScript is not strictly required to track user activity.  There’s still plenty of information available when browsers download images.  At higher levels of risk, JavaScript and then finally plugins like Flash.

A concern today is there is a growing rift in expectations between what consumers think their software applications and services are doing and what’s actually done.  People can handle a few surprises but what happens when the surprises never end?  Trust is the only commodity eroding faster than the US dollar and when it’s gone I’m not sure what happens.  We need a modern day Andy Rooney to crack the whip.


If your a Firefox user there is a new add-on available called Lightbeam.  Lightbeam is useful for understanding how personal data is shared on the Internet like web browsing habits, sites you frequent, etc.  Lightbeam works by recording sites you visit and also recording any included third party sites that may be required by the sites you visit.

Lightbeam does not reveal how companies leverage your personal data for their business uses or if they even store your personal data.  A good general rule of thumb, if someone has the capability to snoop your personal data assume they are.  This way you will not be unpleasantly surprised at the next big privacy headline in the media.

“A good general rule of thumb, if someone has the capability to snoop your personal data assume they are.  This way you will not be unpleasantly surprised at the next big privacy headline in the media.”

To get a better look at Lightbeam, double-click my thumbnail picture (top) to view an enlarged photo.  You will notice in my browsing history, I visited 37 sites which referenced 149 third party sites.   Third party sites are sites included by the site you visited and most likely, many without your knowledge.  Some might argue without consent as well but most of us click through those 60+ page licensee agreements anyway (don’t we).

The Lightbeam user interface allows you to move nodes around, toggle controls on/off, etc.  In looking over my results, some common third parties emerge like Google Adsense and DoubleClick.  Many sites use Google advertising on their pages so nothing too surprising here, we see ads everyday.  However, you may have not considered the implications of third party content on the many pages you visit.

photo: Paros Proxy, HTTP Request

To best illustrate what’s happening between the web browser and server, on the left is a screen fragment from a tool called Paros Proxy.  Paros sits between your web browser and sites on the Internet you wish to view.  When you request site content, Paros intercepts the HTTP request, displays it, and forwards the request on to the server.  Paros facilitates request introspection or even modifies requests en route if you wish.  For our purposes, we are interested in viewing HTTP requests.  In this example, I visited the web site but many sites access Google services.   To begin, requests a third party Google syndication link, first red circle.  In the second link, also circled in red, the web browser specifies a Referer.  The referer is part of the HTTP protocol and sent to the site to specify which page the browser was on before the link was clicked.  Said a simpler way, the web site your navigating to knows the web site you came from.  Often it’s another page on the same web site like switching between tabs on a news site but it could be from an entirely different web site like one of your browser bookmarks.

The concern is that when site content is loaded, the third party site is notified of the site you browsed previously.  In this case, since Google content is ubiquitous so it means Google knows which sites you browsed even if you didn’t get their via their search engine or web browser.  There are many more ways to leak information than the referer so it’s only part of the problem and referer does have legitimate uses.  Cookies and URL rewriting are also combined to make your browsing experience personal or tied directly to you as an individual.

What is or should be private is evolving and everyone has an opinion.  Internet service providers desire more access to end-user personal information.  Individuals are continually surprised to see how their private information is shared between companies.  Whatever your views, Lightbeam provides provides transparency about personal data shared between third parties in a way many can understand.  Lightbeam is released at a time when the public concerns about privacy and transparency are at an all time high.  It will be interesting to see if the tool gains traction with the public.



[Updated Post Friday December 10, 2012]

I thought it would be helpful to share my results with everyone.  Figure 2 is chart showing hourly trending for my Internet speed over the last 2 weeks.

Figure 2: Hourly Trend of Broadband Speed

My Internet connection to my ISP is 50Mbps.  The chart shows I’m receiving slightly less than 1/2 my expected bandwidth.  I changed the chart around to produce bandwidth results in megabits per second (e.g., Mbps).  Megabits is how my ISP advertises their connections so it’s less error prone for me to think of my results in terms of Mbps.  At little Excel magic yields the chart in Figure 2.  Now that measurements and charting are in good shape I’m thinking about my data and results.  I have come to the conclusion it’s best if I rerun the tests connected directly to my ISPs access point.

In my home network my *NIX host, running the testing, connects to a HP Procurve 1700-8 managed switch, then to a Netgear Powerline AV500 Adapter running as a bridge, then to my Asus RT-N56U router/firewall, and finally to the ISPs access point.  It seems likely to me the WAN connection should be the most rate limited of all the gear.  However, to eliminate all possibility of error, I’m going to connect the *NIX host directly to the ISP access point and rerun the tests.  Sigh.

[Updated Post Friday November 23, 2012]

I made a few improvements I thought I would share.  The original program code had interleaving series data in rows.  There’s nothing wrong with the data but it’s hard to graph in Excel.  I improved the program by moving series data to columns.  Once I get all the data into Excel I delete all the columns except: DATE, TIME, ELAPSED-sjc01-10, ELAPSED-sjc01-100, ELAPSED-wdc01-10, ELAPSED-dal01-10.  To graph, I select all the data including column headings and choose the graph I want.  Following are the improved files.

Binary – ispspeedcheck2.jar
Code –

If you run the tool, following are a few lines of sample data you can expect to see in the new CSV file.

11/20/2012,7:14:09 PM,sjc01-10,11536384,8145,sjc01-100,104874307,51949,wdc01-10,11536384,6383,dal01-10,11536384,5972
11/20/2012,8:15:14 PM,sjc01-10,11536384,5295,sjc01-100,104874307,46618,wdc01-10,11536384,6157,dal01-10,11536384,5537
11/20/2012,9:16:25 PM,sjc01-10,11536384,8588,sjc01-100,104874307,49301,wdc01-10,11536384,6273,dal01-10,11536384,5595

The following is a chart I generated with a few hours of data.

Updated Figure 1: Broadband Bandwidth Hourly Trend

Keep in mind if your going to run the new jar you should update your command line like the following.
java -Dfile=./ispspeedtest2.csv -jar ./ispspeedcheck2.jar

I’m interested to see some real data.  Happy Thanksgiving!  –Milton

[Original Post]

Many of us at one time or another wonder if we are really receiving our full Internet bandwidth.  There are a number of ways we can get this information.  I will cover a few of them as well as provide a small Java executable along with source code so you can experiment on your own.

If your only looking for a quick easy test for your Internet connection you don’t need to write a program.  Open your browser to and give them a try.  I like and it’s easy to use.  In my case, I wanted an hour by hour trend of my Internet connection throughput.  To get a trends on speedtest would be bothersome since I would have to run the test manually once each hour.  Great tool but you need a little something more if your interested in trends.  This monkey has better buttons to push so I decided on a Java project.  Moving on, following are some of the questions and points I considered when thinking about my broadband bandwidth.

Am I receiving my full Internet bandwidth?
Broadband is prickly business.  On the one hand, Internet Service Providers(ISPs) advertise and sell their connections by bandwidth.  On other hand, they don’t make any guarantees about the bandwidth they provide.  As a consumer, what am I really receiving?  The impact to my wallet is guaranteed, that happens regularly each month, but my bandwidth is variable.  To be an ISP is a great line of business.  ISPs can command a constant monthly price for a product they may deliver in more or less in quantity — at their sole discretion.  Imagine a grocery store operating on the same principle.  You pull up to the checkout counter with a full shopping cart of groceries or 1/2 a cart and it makes no difference.  It’s all the same price.  Ok, I feel a rant coming on so I’ll stop now but you get my point and source of my curiosity.
Paying for additional bandwidth provides more bandwidth but how much more?
I’m not sure how to answer this question at the moment, at least without changing my service.  Perhaps data on or similar sites.  For instance, if my connection is 50Mb/sec download and I’m receiving 25Mb/sec will upgrading to 128Mb/sec produce half as much throughput or 64Mb/sec on average?  Likewise, if I cut my bandwidth to 25Mb/sec do I really get half as much throughput or is it only 20% of the advertised capacity for slower speeds?  My assumption is that higher speed connections have more wiggle room for throttling whereas the lower speed connections are closer to the advertised throughput.  I think these questions are more suitable for an ISP review as opposed to a broadband tool measurement discussion but it’s still something I’m interested to know.

Is there more Internet bandwidth available during off-peak hours?
It seems common sense during early morning or late evening hours more bandwidth is available since less people in your region are surfing the Internet.  Of course, does it really matter if there is Internet bandwidth when nobody wants to use it?  If a tree falls in the woods… ah, never mind.  Ok, unless your a torrent freak or schedule off peak downloads there’s probably nothing practical in this line of thinking but I’m curious to find out.
Complexities of bandwidth measurement
Measuring your throughput is easy.  Understand the reasons for the numbers are more complicated.  There are many factors that influence overall results.  I don’t claim to be a network expert but following are some areas that come to mind.

WAN Speed – Yay.  Generally, the broadband speed rating is between you and your ISP.  There is absolutely no guarantee to other servers on the Internet.  So the at best, you will only receive your advertised throughput to your ISP.  Traffic anywhere else may take the slow boat.
LAN Speed – Are the kids playing games or downloading content over the same connection?  If so, this will impact any measurements.  Also any IP enabled devices that share the LAN, television sets, smart phones, almost anything.  Even thermostats are IP enabled (incidentally NEST is really cool – Google it).  You could have significant traffic on your LAN interfering with your results.
WIFI – Are you working over WIFI?  Limited WIFI bandwidth or poor signal quality will negatively impact your results.  Better to be connected directly Ethernet.
Network Hardware – All boxes with blinky lights are not created equal.  I’ve tried many brands and settled on a highly rated ASUS RT-N56U router recommended on SmallNetBuilder.  One word — awesome!  The value of good network hardware cannot be understated.  Don’t assume a popular brand is the best.  In the age of the Internet, lots of people can make the same mistakes.  Don’t follow the sheeple.  Do your own homework.

Traffic Shaping – ISP’s shape traffic all the time.  Some have been accused of throttling peer to peer traffic like bittorrent.  It’s almost guaranteed if your running speed tests against popular sites like your ISP optimizes that traffic.  Meaning, your throughput to is great but perhaps not consistent across other web sites.  On the surface, traffic shaping sounds a little dubious but the practice has it’s place.  Even in the home, most consumer devices support some amount of traffic shaping.  A really comprehensive test would probably test traffic over several different protocols.  A test over different protocols would help work around traffic shaping.  I’m not going to do this just yet but it might produce a better result.  Electronic Frontier Foundation has some interesting resources on shaping[2].
Broadband history
Taking a reading and tossing it over the fence to your ISP is not very convincing if you have bandband throughput concerns.  Too many factors, as mentioned previously, can influence a short speedtest run and produce misleading results.  On the other hand, approaching your ISP with a 2 month history of hour-by-hour Internet bandwidth trend data is compelling evidence.  Even if you don’t have a case to prove, more comprehensive information is always desirable.  What is the average WAN usage over time by hour of day?  The mean, median, standard deviation of download times for different size payloads?  Are 100Mb payload download times one order of magnitude greater than 10Mb or different?
My environment
I have several computer systems at the house.  In my home we have at least two computers per person.  We also have a cadre of WIFI enabled devices like Kindle Fire, iPhones, IP printer, VOIP devices, flatscreen TV, etc.  We use several operating systems: OSX for business, Ubuntu Linux for special tasks, and Windows for gaming.  I develop on my OSX system using Eclipse and deploy longer running tests on my Ubuntu server which works tirelessly nights and weekends without interruption.
The experiment
I thought about adding lots of bells and whistles to the program code.  Adding a user interface, fancy logging, etc.  In the end, I chose to keep the code very simple.  The output of the program is a simple CSV file that can be loaded into Microsoft Excel.  Following are a few lines of output from the CSV log file.

milton@sparky:~/bin$ tail -f ./ispspeedtest.csv
11/14/2012,8:31:13 PM,sjc01-10,11536384,6355
11/14/2012,8:32:15 PM,sjc01-100,104874307,61625
11/14/2012,8:32:23 PM,wdc01-10,11536384,7625
11/14/2012,8:32:31 PM,dal01-10,11536384,7210
11/14/2012,9:32:37 PM,sjc01-10,11536384,6469
11/14/2012,9:33:37 PM,sjc01-100,104874307,60103
11/14/2012,9:33:45 PM,wdc01-10,11536384,7121
11/14/2012,9:33:52 PM,dal01-10,11536384,6979

The first line is the command line.  I use Linux tail command to show the log realtime in a command shell window.  Following the command line are, column headings, date and time message was logged, server code the payload was downloaded from, payload size in bytes, and the download time in milliseconds.  For instance, the first line of data, the payload is downloaded from sjc01-10 (a code for a speedtest server in SF Bay Area).  The size of the payload was 11.5Mb and it took 6.3 seconds to download.  The longer the program runs the more data is produced.  The tests are single threaded.  Once you have collected enough data you can stop the program (e.g., CTRL-C) or the program will stop on it’s own after 1000hrs with the default settings.
Figure 1 shows a couple of days of hourly broadband trend data.  The top red line is the hourly data for a 100Mb zip file payload.  The blue line on the bottom of the figure is the hourly data for the 10Mb zip file payload.
Figure 1: Broadband bandwidth hourly trend
I realize the chart is small and not easy to read.  I provided the chart to give you an idea of what a finished chart may look like when it’s done.  Unlike the stock market or your 401k, a line trending down is good and means your downloads are taking less time.
Source code, binaries, and limitations (and dragons)
Following is a sample source code package as well as executable jar if you don’t want to compile.  Keep in mind the program is not full featured and not suitable for production systems.  I have provided no security controls like input validation.  The program is not multi-process safe or even thread safe in some places if you choose to repurpose some of the classes.  I originally made this for myself.  Thar be dragons!  Consider yourself warned.

The servers where I download payloads are hard coded in the program for North America only.  If you live elsewhere, you will want to experiment with ISPs closer to your region and you will need to make some improvements to the code.

Code –

Running the binary depends upon your system but to get it running on my system I use the following.  The command launches the program and tells it to log CSV data to the ispspeedtest.csv file in the current working directory.
java -Dfile=./ispspeedtest.csv -jar ./ispspeedcheck.jar

I have tried it under Java 6 and Java 7 on OSX and Java 6 under Ubuntu Linux.  If anyone needs some assistance let me know and I might be able to make some changes or suggest some.  I’m just not sure who’s interested in this stuff at the moment so I see need to invest anymore time than necessary to help answer my personal curiosities.

Alternative ideas
Writing a program to measure your Internet throughput is probably not the most efficient use anyone’s time.  My first thought was to use Linux Curl command or WGET command to download the payloads.  Then use a graphing facility like RRDtool.  I’m not too familiar with RRDtool but it looked powerful and promising.  If your a Linux user a little Googling and hammering out a shell script is a good way to go.  I did some Googling to find desktop applications.  I didn’t find anything for OSX.  I did find an interesting package for Windows.  To be honest, I just wanted to do this in Java.

Contacting your ISP
Before you decide to share any information with your ISP make sure you have your facts straight.  Double check your information, cross check with other tools, etc.  It’s easy to misinterpret results or make a programming mistakes if you decide to make some improvements to the code.  If you’ve taken all precautions and you still wish to contact your ISP then be courteous.  Often raising awareness will get you on the path to resolution.  Also keep in mind your advertised bandwidth is not a guarantee.  Bandwidth guarantees are available but not generally to consumers since they cost a fortune.  Take the time to become familiar with your ISP’s Terms of Service for your Internet connection before you call.

Verdict and next steps
No real results just yet.  I’m still getting started and I only have a couple of days of data.  I will update in the future with my results or describe how the program works if there’s interest.

Good luck.

[1] Alves, J. Clipart – Wavy Checkered Flag. Digital image. Clipart – Wavy Checkered Flag., 25 Mar. 2010. Web. 15 Nov. 2012. <>.

[2] “Test Your ISP.” Electronic Frontier Foundation. EFF, n.d. Web. 15 Nov. 2012. <>.