LocalTV+

There was a lot happening behind the scenes. Today there will be new iOS and tvOS betas released with notable changes. Everyone should notice that videos start up much faster. We have removed many delays that prevented video from starting as soon as you click on a show. There is a major UI update for tvOS that should be a much better user experience.

On iOS you will notice two new buttons below a show description. The share button lets you share information about the show with others either via chat or social media. The bell button allows you to set a reminder for the show. A clear bell means you can set a reminder and filled in bell indicates a reminder has been set. Since are very serious about privacy so this information never leaves your device. As always, we do not track what shows people watch or want to watch. A few min before the show starts you will receive an alert on your phone.

Finally we would like to thank all of the donors who have contributed to this project. If you are interested in donating, please use this link: https://givebutter.com/LocalTVPlus

As a matter of principle, we want to retain as little information as possible about our users. In order to minimize friction and retain almost no personally identifying information, we have rolled out Sign in with Apple. When you use this we recommend you pick the ‘hide email’ option. This way, you have an account on our system but we do know anything about you other thans the fake email address Apple provides. This address forwards to your real email and still allows us to contact you if we ever need to.

Thank you everyone who used the app to watch the Oscars last week. As expected the service ran without a hitch. Also, as expected, concurrent viewer numbers were not quite the Super Bowl but still more than we usually see. At its peak there were 37 concurrent viewers watching. I see this as more of an indicator of a wider user base from where we started.

The CDN continues to perform as expected. You can see we served the same amount of data (50gb) but the cache served another 130gb of data

We have launched our website. While this blog will continue with engineering details, please use https://www.yourlocaltv.org going forward when telling people about this service.

We have doubled our tester count to over 1000. There is very little concurrent use however, so we don’t have enough data to understand what the breaking point of this architecture is. There are a couple of major events coming up where I hope people use the app and help us gather data.

1. Academy awards : Sunday March 10th on ABC
2. NCAA March Madness: CBS starting March 17th and will broadcast all month.

As always, the app is free to use as much as you want. If you can, please consider donating. If you can’t donate, spreading the word or just using the app helps us improve the service.

Based on the feedback from users on reddit as well as my own experience, the Super Bowl test seems to have passed with flying colors. Everyone seems to have had a perfect video stream. The new antenna has had perfect signal. Additionally, the switch from RCN to Boston Broadband as our ISP is clearly visible in the response time graph below. Given this is a real time system, latency is important and the faster responses have helped.

We hit some of our highest concurrent numbers. Of the 400 or so testers that were using the app at the time, 65 were watching the Super Bowl. We were able to service everyone. For the Super Bowl we pumped out 360GB of data from our CDN. Based on how the CDN performed, while these are tiny numbers I am more confident the current stack should scale up to much more. The generally flat green line while the blue spikes indicates the CDN is doing a much better job of absorbing almost the entire load. We have increased the test flight limit to allow more users to use it. Thank you everyone who watched with the app. Please keep using it as the data we get from these tests allows us to keep improving our software.

The purpose of this long test phase is to keep making adjustments so we have a service that can reasonably handle the volume to traffic we expect while still keeping operating costs low.

Looking at the Super Bowl you can clearly see when the game started, ended, where and when the game got more interesting.

Finally, some house keeping. If you are new to this blog, this is primarily an engineering blog for the free LocalTV+ service run by Mass Local TV, a non profit based in Boston. This is a temporary website. Some quick answers to common questions:

1. The app is only for iOS an tvOS for now. There are other versions in development. You can get the test app on TestFlight.
2. The service only works 100 miles around Boston.
3. We are a non profit and donations are tax deductible. You do not need to donate to use the service. It does, however, help keep the lights on
4. This service was made by three people who are all volunteers.
5. Join us on reddit, which doubles as a support forum.
6. If you have GitHub you can submit bugs and feature requests.

The Super Bowl on Feb 11 will be the largest test of this platform. We have seen large numbers during the NFL playoff games but we expect the Sunday to be the biggest traffic spike yet. This will be a trial by fire for this service. In preparation for that we have rolled out some server and app updates. We have also moved our hosting to Boston Broadband, a great locally owned ISP you should check out. At the new location, we now have an antenna with a clear line of sight to the towers in Needham. This will provide us with clear reception on all channels regardless of weather conditions. The new antenna and hosting was funded by donations from beta testers who have found the app useful. LocalTV is funded by viewers like you. Thank you. We would like to stay free for as long as we can. If you find this useful please consider making a tax deductible donation. Depending on employer, your donations may be matched. If your employer uses Benevity you can find us there for easy donation and matching.

The new antenna in Brookline

If you would like to test LocalTV+ on iOS or Apple TV. You can use our public testflight link.

With the new location users who have been using the app for the past few months will notice several other changes:

1. PBS (WGBH) is now in high definition
2. FOX is much more reliable and we are trialing subtitle support
3. CBS works
4. Overall much faster and hopefully little if any freezing

Over the Thanksgiving, we ran our largest public beta yet. Expanding from a dozen users to over a hundred. This was announced on Reddit Boston . This was an excellent learning opportunity that stress tested the service.

As expected, increasing the concurrent traffic volume by an order of magnitude revealed bugs. Thanksgiving day football traffic caused multiple failures that we were able to investigate and improve. A lot of the issues related to the configuration of the CDN. The initial configuration had a low cache hit rate (10%) which manifested in much higher server load further down the stack. LocalTV+ is built as a collection of micro services, which can be scaled up or down as needed to optimize for cost/traffic. The cache failure rate began to increase load on other services which eventually became high CPU load on a service I never expected high load on.

We made changes to the CDN on Friday and in time for college and NFL games over Saturday and Sunday. This was a much more successful test. The cache fixes worked and the load was moved to the CDN. Overall when the signal was ok, almost everyone was able to watch the game with little to no issues. The issues that were reported were the infrequent freezing and re-buffering for a few seconds. We will continue to test and if the performance is consistent we will be able to open up the beta testing to more users.

Over the course of the weekend we also opened up a dedicated subreddit for general discussion and feedback.

Other findings:

1. Several crashing bugs related to tvOS were identified
2. Persistent errors on first login / registration about not being able to get streaming links
3. The need to improve onboarding and the UI in many places e.g. highlighting on tvOS
4. Airplay to non Apple devices work most of the time but is inconsistent.
5. CBS continues to be a problem. This is related to the current antenna location.
6. We will publish an FAQ soon. This will answer many common questions. e.g. Android (Yes, but will be later) or closed captions (Yes, it is just currently disabled while testing)

I realize that we have never shared images of the AppleTV UI. Oddly, while iOS gets more use, this was the original app with the iOS port coming later. The TV UI still needs a lot of polish, but it already feels quite pleasant to use. Just like iOS, it support picture in picture.

LocalTV+ is currently available for beta testers in Boston. If you aren’t currently a tester you may be wondering what the app is like. We are aiming for an app that feels like it is native to iOS with a user experience that doesn’t hate you. The video below shows how you can quickly switch from show to show and utilize picture in picture to go to other apps without interruption.

Cable is expensive and an entire generation does not know that over the air TV (NBC, CBS, PBS, FOX etc.) is free with a cheap antenna. People think it’s an illegal hack despite this being how people have gotten TV for generations. Additionally, a lot of people live in areas and homes served by these stations but do not receive a good signal due to environmental obstruction such as trees or hills. US law has a provision for this to allow non profits to retransmit the signal to resolve this issue. There was an attempt by Locast to do this recently but it was shut down not due to the retransmission, which everyone agrees was legal, but the way they monetized and used their money.

LocalTV+ is an attempt to provide unmodified TV signals for certain markets in the US over the internet. It should not be necessary for people to buy expensive plans on streaming providers or go to pirate websites to watch signals that are free. Viewing a show on a phone should be as straight forward as watching something on Netflix. You also should not need to hunt down which balkanized streaming service a popular show or sport is airing on.

LocalTV+ is currently in private beta for iOS for just the Boston market. There will be a TvOS version soon. Watch this site for updates.