MobiledgeX Edge-Cloud 3.0: Enhanced GPU/vGPU Support

September 8th, 2021

Headshot for Ashish Jain

Ashish Jain

Software Engineer

Today, many applications rely on GPUs for accelerated image processing, graphics, video rendering, AI modeling, and more. For applications optimized for the edge, minimizing both network and processing latency is paramount.  With GPUs at the edge, an application developer has the benefit of deploying an application backend that has lower processing time with lower network latency. This provides the best user experience for applications on mobile devices over a cellular network.

It is very difficult and time-consuming for application developers to manage dependencies that exist between GPUs, GPU drivers, hypervisors, and operating systems as well as for the operators who have to maintain these environments.

For operators and developers, the MobiledgeX Edge-Cloud platform acts as a bridge to simplify the management of GPUs, making them easily accessible at the edge.

With the release of MobiledgeX Edge-Cloud 3.0, we've introduced new GPU and vGPU management features, both for operators and developers.

Operators will have simplified management of GPU drivers for their cloudlets. Our services will take care of installing appropriate drivers on the developer’s cluster VMs and associate appropriate license (if any) with that driver. We support managing GPU PCI-Passthrough drivers and vGPU drivers along with maintaining their respective license configurations. Operators are also provided with an insight into the usage statistics of the GPU resources on each cloudlet.

Developers can use any of the GPU flavors for their application, providing access to GPU resources. It is completely automated for various application deployments with enough flexibility for customizations. Developers are also provided with an option to download the GPU driver, in case they want to set it up themselves.

Simplified Management For Operators

Operators have to deal with setting up GPU hardware from a vendor, virtualizing it for various use cases, and managing license servers to enable applications to use that hardware. Although hardware setup is a one-time process, managing GPU drivers is not. Furthermore, maintaining support for various deployments like Kubernetes, Docker, VM, etc is complex. MobiledgeX simplifies this for operators.

Our software helps operators manage and automate the installation of GPU drivers and setup license configs for various application deployments. An operator is also provided with control over who can access those drivers. MobiledgeX supports simple APIs to manage GPU drivers of different types like PCI Passthrough and vGPUs and also maintain driver builds based on different versions of GPU drivers, hypervisors, kernel versions, operating systems, etc.

In the MobiledgeX Edge-Cloud Console, GPU drivers and their license configurations can be managed from the “GPU drivers” section as shown in the following screenshot:

Figure 1: List of GPU Drivers

Multiple builds of the same GPU driver can be maintained for various operating systems, hypervisors, kernel versions, etc. Here is an example of how to create a GPU driver from the console and add various builds to it:

Figure 2: GPU driver creation

Once a driver is created, it will be visible on the “GPU drivers” section. Detailed information of the driver can be seen by clicking on the driver from the list provided. Here’s the detailed information of the driver we just created:

Figure 3: Details about the GPU driver

As seen above, MobiledgeX Console provides enough flexibility to support multiple builds for a single GPU driver depending on the VM specification. All the drivers and their license configurations are cached in secure storage. Operators and MobiledgeX admins will control the access to that storage. Operator created GPU drivers are only accessible by users belonging to the operator organization and developers part of the operator’s cloudlet pool. MobiledgeX created GPU drivers are accessible by all the users.

Operators can then associate a GPU driver with a cloudlet either during create or update of the cloudlet. On the MobiledgeX Console, this can be done as part of “Advanced Settings” inside the cloudlet create or update page:

Figure 4: Associating GPU driver with a Cloudlet

Once a GPU driver is associated with a cloudlet, MobiledgeX services will automatically select and install matching drivers for various application deployments like Kubernetes, Helm, Docker, and VM. It will also automate license configuration wherever applicable.

Operators are provided with statistics on the count of GPUs used across their cloudlets. This gives operators a better insight into the usage pattern of GPUs that can help with cost optimization. This can be viewed in the Monitoring section of the MobiledgeX Edge-Cloud console:


Figure 5: GPU usage statistics

Simplified Accessibility For Developers

Ideally, an application developer never has to deal with managing the complexities of the underlying infrastructure and can focus primarily on their application code. For that reason, dealing with GPU drivers shouldn’t be a burden for the developers. We simplify this to great extent for diverse application deployment types and for various hypervisors and operating systems.

Once a GPU driver is associated with a cloudlet by the operator, along with the license config, a developer will only have to select a GPU flavor, and their application will have access to the GPU resource. Developers will not have to worry about setting up a GPU driver or setting up a license config for non-VM based applications. The goal is to simplify deployments for developers so that they can focus only on the application definition and scalability.

To enable access to a GPU resource, a developer will have to deploy a cluster instance or auto-cluster instance using a GPU flavor. More information on application deployment on a cluster instance can be found in this Deployment Essentials guide.

In the  MobiledgeX Edge-Cloud console, the “Flavors” section shows all the supported flavors including GPU flavors:

Figure 6: List of flavors including GPU flavors

As seen above in this example, there are two GPU flavors: x1.medium.gpu and x1.large.vgpu. More details about the flavor can be found by selecting the flavor from the above list:

Figure 7: Details about the GPU flavor

The Flavor details page provides additional information about the GPU flavor, for instance, in the above output x1.medium.gpu is a single instance of NVIDIA-T4 PCI Passthrough GPU. Similarly, x1.medium.vgpu is a single slice of vGPU hardware. As seen above, a developer can select from various types of GPU / vGPU hardware.

A developer can then use this flavor to enable access to GPU resources for their application. For example, the following page on MobiledgeX Console creates a Kubernetes cluster instance with GPU support:

Figure 8: Cluster Instance creation using GPU flavor

After creating a cluster instance, an application can be deployed on it.

Figure 9: Application Instance creation using GPU flavor

For Docker/Kubernetes/Helm based deployment, one additional step is required from developers if the developer uses a custom deployment manifest as part of the application definition. More information on that can be found in this Guide To Deploy GPU Applications.

In the case of VM-based applications, a developer would require access to GPU drivers for their custom VM images. Developers can access any MobiledgeX provided GPU drivers and any operator provided drivers (if they are part of that operator’s cloudlet pool) from the details page of the GPU driver on MobiledgeX Console:

Figure 10: Option to download specific GPU driver build

Users are provided access to the drivers which they can select based on parameters like kernel version, hypervisor, and operating system. To get the download URL of a specific build, one can click on the link icon as highlighted in the above screenshot. This opens a dialog box with details on the build download URL:

Figure 11: Details on how to download a specific GPU driver build

There are two options to download the URL:

  1. cURL command: This is mainly intended for those users who want to download the GPU driver build inside a remote VM

  2. Direct download: To directly download the build on the local machine

The provided download URL is only valid for the duration mentioned on the above page, in this case, it is 20 minutes. Overall, developers are provided access to drivers which they can select based on parameters like kernel version, hypervisor, and operating system.

Conclusion

With the increased demand for GPU support and access to edge sites for lower latency requirements, it is crucial for an edge cloud to support and simplify the process to manage and access GPU resources. MobiledgeX does simplify this to great extent for both developers and operators. Operators are provided with simple APIs to set up drivers for their cloudlet. And developers can access provided GPU resources by just selecting an appropriate GPU flavor.

GPU driver management is also supported using our command-line utility and REST APIs.

To learn more, visit the MobiledgeX Developer and Operator portals. 

We would love to hear back from you. For feedback or questions, head to our Getting Started page on the Developer Portal and sign up for a time to chat with our Developer Advocacy team.