Finding the Edge at Scale
September 25th, 2018
In our first blog, Jason wrote about our coming out at Mobile World Congress Americas (MWCA) and he talked about what MobiledgeX was doing as a business at a high level, and why that makes good sense. The week after MCWA we participated in the Edge Congress in Berlin and it was my turn to talk a little about how we are doing this (a few more engineering and architecture details) and to even show an actual Android application running with MobiledgeX augmentation on the Deutsche Telekom German infrastructure. It was a real demo (not slideware or screenware)—real software running on real cellular infrastructure. I’ll give a few more details after setting the context (spoiler alert: the big story is how to deal with the scope and complexity of the mobile infrastructure).
It’s easy to forget the scope and complexity of the global cellular infrastructure because it does what most computer technology can only aspire to—it just works and does so transparently. As an engineer and system architect, that tends to leave me breathless if I think about all the mechanisms it takes for me to get off the plane in Berlin, turn my phone on, use it and get billed for what I do. The cellular system is built out of the same digital computer and network technologies that computer people use (including a lot of software running on standard servers), but there is a lot more to it than most suspect. Let me put it this way—the global cellular infrastructure represents more than 1.7 trillion dollars of investment in the past 10 years. In contrast, the public cloud giants have spent 300 billion dollars over a similar time period. There are nearly 800 mobile operators globally, connecting more than 5 billion people today [https://www.gsma.com/mobileeconomy/wp-content/uploads/2018/02/The-Mobile-Economy-Global-2018.pdf ] and with a forecast of 3.5 billion addition IOT connections by 2023 [https://www.ericsson.com/en/mobility-report/reports/june-2018/iot-connections-outlook ]. Both people and machines are connected at the edge via more than 6 million cell towers [https://www.mobileworldlive.com/blog/blog-global-base-station-count-7m-or-4-times-higher/ ].
I’m repeating all these statistics because you need to hold them in mind when you consider what MobiledgeX is doing. When we’re fully deployed (many mobile operators, not just Deutsche Telekom) and when we’ve built a successful marketplace to provide edge augmentation services, when a smartphone user lands in some city around the globe, turns on their phone and starts to use one of the many MobiledgeX-enhanced applications, our infrastructure will have to perform quickly at global scale, as transparently as the cellular system places phone calls today. From the first time I thought about this opportunity, I thought of the issue of scale and when I talk to my engineers or our marketing team it’s the same reminder—it’s all about doing this at scale.
Our Approach to Edge
Let’s double click down on what we’re doing. We believe there is high value in being able to execute various services near the edge of the mobile network (conceptually next to the radio network that the phone connects to). These could be functions that take advantage of the untapped capabilities of cellular services (e.g, better knowledge of who you are and where you are), that provide new edge services in the cellular network (e.g, CDN proxies), that augment the capabilities of the phone for demanding A/R and V/R applications, or enable high-speed collaboration between groups of nearby users playing games, for example.
We do that based on having formed business relationships with a large set of mobile operators around the world (there are many MobiledgeX values that accrue to the mobile operator as well as the device user and application owner—we believe we have something for almost everyone). These operators agree to give us access to excess server capacity (just like roaming today is the sharing of excess telephony capacity) between operators. In contrast to Internet ISP’s, the mobile operators operate lots of standard servers to run their businesses (technically and commercially). All we need to do our bit is access to some standard infrastructure (e.g., OpenStack or VMW) that can create containers or virtual servers as needed, that we can discover and use. We run software on specific created instances to implement our distributed control structure and to provide services in support of a mobile user or application. We pay the mobile operators for what we use, and recover that in the fees we charge
Our goal is to do this without requiring any participant (mobile operator, application or service owner) to change its business model or mode of operation.
To do this our software has to perform an impedance match between two very different worlds and technologies: to a developer or application owner we need to be friendly, dynamic and cloud-like, and our marketplace (the collective community of our developers) has to offer compelling functionality for important applications. The mobile operators need to trust us as a business partner and trust us technically not to destabilize the global cellular infrastructure. Our goal is to do this without requiring any participant (mobile operator, application or service owner) to change its business model or mode of operation.
Edge Services We’re Working On
There are many ways in which the MobiledgeX infrastructure can be used commercially. In the case of a MobiledgeX-augmented smartphone application, it typically happens this way. The application developer includes a MobiledgeX library in their software build. When the application runs the application and device/subscriber register with our global control infrastructure. The registration process identifies the “backend” software the application would like to use (although in this case it’s more like “edgeend.”) The MobiledgeX system then determines an optimal location to run that backend from the set of operator partner locations, and deploys and instantiates that backend, all in the veritable blink of an eye.
We call the subsystem that does this the Distributed Matching Engine, a microservices-based distributed component of MobiledgeX. The initial version is up and running on Google Cloud, written in such a way that we believe it can scale in capacity and distribution to meet our ambitious objectives. We’re not going to say a lot more about the details, but those of us who have been my distributed systems colleagues at Sun and then Pluribus can guess at some of the bits. In the end, we think this fabric would speak for itself just by doing what it’s designed to do, but that demonstration will be continuing rolling thunder as we sign up more operators and support more applications and give more demos. Stay tuned.
The other major components of our system include a geographically distributed, microservices based, controller to manage the developers, developer applications and operator provided cloudlets and a microservices based Cloudlet Resource Manager (CRM) that manages individual cloudlets while being agnostic to the IaaS layer the operator used underneath. On top is a micro-services based edge-cloud controller that can be country specific (to deal with country specific privacy laws) and deploys application backend on cloudlets based on demand seen for that particular application by the Distributed Matching Engines in each location. Obviously the system has to deal with more complexity than that but this should give the idea at a high level and in future blogs I will go into more details.
Quick Glimpse of the Edge demo
Oh, at the beginning I promised to describe our demo briefly, so here goes. We have a working version of the initial libraries that need to be included in an Android application (IoS is still under development). Using the library, we built a simple application where the user can say what his location is (or use the location that the phone already has calculated). When the application has begun, the device (and user) are registered in the MobiledgeX control system, and we find an optimal location to execute the backend component of the application. When the user submits his location, we use our integration with Deutsche Telekom’s cellular infrastructure to verify that the location claimed is consistent with the location seen by the phone system (which cell is being used). If those answers disagree, the location claimed is declared to be fraudulent.
As any engineer knows, it feels wonderful to see your technical baby take its first steps and I’m very proud of what the team has done in a short period of time, but I’m even prouder of the fact that this simple demo is running on an early version or our infrastructure that can scale to match the modern global cellular infrastructure.