Niantic CEO urges AR industry to learn from mistakes of today’s tech
John Hanke, CEO of Niantic—the company behind Pokémon Go—recently addressed developers at the AR convention Augmented World Expo 2021.
Today, the tech industry is facing its most important moment in 20 years, a moment in which decisions are going to be made that affect the future of technology—and humanity.
That was the message John Hanke, CEO of Niantic, gave AR and VR developers attending Augmented World Expo 2021 last week. Niantic is the company behind Pokémon Go and other augmented reality apps, and is currently developing AR glasses.
Hanke was, of course, talking about the coming metaverse, the term pushed center stage by Facebook (now Meta) as a description of the era of wearable, always available, augmented reality technology. The term metaverse comes from the futuristic 1992 novel Snow Crash by Neal Stephenson.
“I love [Stephenson’s] work,” said Hanke, “but we all know how those stories go. In the fictional future, the world has become such a mess, things have gone so horribly wrong, that people have to escape into a virtual reality.”
“That’s not the future I want,” Hanke said, “it’s not the future we believe we will happen. We think we can use this tech not to escape the world into VR, but to build a better real world, preserve the real world as a place of purpose and novelty and community.”
Company executives and developers, he said, need to now focus on building the metaverse responsibly, learning from what has been going on in tech industry for past ten years—including the spread of misinformation, political division, and loss of privacy.
“As excited as we are about the technology and what we can do with it, we recognize that this next phase of computing, this platform transition, carries with it big-time responsibility,” Hanke told the developers, gathered in person at the Santa Clara, Calif., convention center and remotely online. “We are going to be making decisions about what this looks like, in terms of platform and product, and about how much human values come into play.”
Hanke looked back at the last big platform change—the emergence of the smartphone.
“We put a small computer in the hands of billions of people around the world, and it changed how all of us lead our lives,” he said. “It’s been transformative for humanity. But we’ve seen the downsides, in terms of tracking, privacy, and the behaviors that the platforms and applications encouraged in us.”
The metaverse, Hanke says, will potentially have all the problems we have now with social media. But, because it will combine with the real world, developers will have even greater responsibility to protect users from these downsides.
“Think about a wearable device, with you all day long, on your head,” Hanke said. “It probably knows where you are looking most of the time. Maybe it knows other things, like your heart rate. If you see an advertisement, did your heart rate go up? When you see a person [it identifies], what did your heartrate do? Did your pupils dilate? What about your emotions? Are you happy, sad, anxious?”
“This is not science fiction,” he continued. “Tech can do what I described. Whether we allow that to evolve into the dystopia that we all know it could be, or take steps to turn it into something else, is a collective job for all of us.”
Tekla S. Perry is a senior editor at IEEE Spectrum. Based in Palo Alto, Calif., she’s been covering the people, companies, and technology that make Silicon Valley a special place for more than 30 years. An IEEE member, she holds a bachelor’s degree in journalism from Michigan State University.
I hope the Metaverse is not corrupted by the profit motive like the online game “Metatopia” described in the novel “Fatal Entanglement,” where the game secretly psychoanalyses the players to find targets to exploit.
This wearable device senses blood glucose and administers insulin accordingly
The Tandem insulin pump, no bigger than a mobile phone, infuses insulin under the skin at the command of Control-IQ software, which has received blood-glucose data from a Dexcom G6 sensor.
In some ways, this is a family story. Peter Kovatchev was a naval engineer who raised his son, Boris, as a problem solver, and who built model ships with his granddaughter, Anna. He also suffered from a form of diabetes in which the pancreas cannot make enough insulin. To control the concentration of glucose in his blood, he had to inject insulin several times a day, using a syringe that he kept in a small metal box in our family’s refrigerator. But although he tried to administer the right amount of insulin at the right times, his blood-glucose control was quite poor. He passed away from diabetes-related complications in 2002.
Boris now conducts research on bioengineered substitutes for the pancreas; Anna is a writer and a designer.
A person who requires insulin must walk a tightrope. Blood-glucose concentration can swing dramatically, and it is particularly affected by meals and exercise. If it falls too low, the person may faint; if it rises too high and stays elevated for too long, the person may go into a coma. To avoid repeated episodes of low blood glucose, patients in the past would often run their blood glucose somewhat high, laying themselves open to long-term complications, such as nerve damage, blindness, and heart disease. And patients always had to keep one eye on their blood glucose levels, which they measured many times a day by pricking their fingers for drops of blood. It was easily the most demanding therapy that patients have ever been required to administer to themselves.
No longer: The artificial pancreas is finally at hand. This is a machine that senses any change in blood glucose and directs a pump to administer either more or less insulin, a task that may be compared to the way a thermostat coupled to an HVAC system controls the temperature of a house. All commercial artificial pancreas systems are still “hybrid,” meaning that users are required to estimate the carbohydrates in a meal they’re about to consume and thus assist the system with glucose control. Nevertheless, the artificial pancreas is a triumph of biotechnology.
It is a triumph of hope, as well. We well remember a morning in late December of 2005, when experts in diabetes technology and bioengineering gathered in the Lister Hill Auditorium at the National Institutes of Health in Bethesda, Md. By that point, existing technology enabled people with diabetes to track their blood glucose levels and use those readings to estimate the amount of insulin they needed. The problem was how to remove human intervention from the equation. A distinguished scientist took the podium and explained that biology’s glucose-regulation mechanism was far too complex to be artificially replicated. Boris Kovatchev and his colleagues disagreed, and after 14 years of work they were able to prove the scientist wrong.
It was yet another confirmation of Arthur Clarke’s First Law: “When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.”
In a healthy endocrine system, the fasting blood glucose level is around 80 to 100 milligrams per deciliter of blood. The entire blood supply of a typical adult contains 4 or 5 grams of sugar—roughly as much as in the paper packet that restaurants offer with coffee. Consuming carbohydrates, either as pure sugar or as a starch such as bread, causes blood glucose levels to rise. A normally functioning pancreas recognizes the incoming sugar rush and secretes insulin to allow the body’s cells to absorb it so that it can be used as energy or stored for such use later on. This process brings the glucose level back to normal.
However, in people with type 1 or insulin-requiring type 2 diabetes—of whom there are nearly 8.5 million in the United States alone—the pancreas produces either no insulin or too little, and the control process must be approximated by artificial means.
In the early days, this approximation was very crude. In 1922, insulin was first isolated and administered to diabetic patients in Canada; for decades after, the syringe was the primary tool used to manage diabetes. Because patients in those days had no way to directly measure blood glucose, they had to test their urine, where traces of sugar proved only that blood-glucose levels had already risen to distressingly high levels. Only in 1970 did ambulatory blood-glucose testing become possible; in 1980 it became commercially available. Chemically treated strips reacted with glucose in a drop of blood, changing color in relation to the glucose concentration. Eventually meters equipped with photodiodes and optical sensors were devised to read the strips more precisely.
The first improvement was in the measurement of blood glucose; the second was in the administration of insulin. The first insulin pump had to be worn like a backpack and was impractical for daily use, but it paved the way for all other intravenous blood-glucose control designs, which began to emerge in the 1970s. The first commercial “artificial pancreas” was a refrigerator-size machine called the Biostator, intended for use in hospitals. However, its bulk and its method of infusing insulin directly into a vein prevented it from advancing beyond hospital experiments.
The original artificial pancreas, called the Biostator, is shown here in hospital use in about 1977. It delivered insulin and glucose directly into the veins and could not be adapted to home use.William Clarke/University of Virginia
That decade also saw work on more advanced insulin-delivery tools: pumps that could continually infuse insulin through a needle placed under the skin. The first such commercial pump, Dean Kamen’s AutoSyringe, was introduced in the late 1970s, but the patient still had to program it based on periodic blood-glucose measurements done by finger sticks.
Through all this time, patients continued to depend on finger sticks. Finally, in 1999, Medtronic introduced the first continuous glucose monitor portable enough for outpatient use. A thin electrode is inserted under the skin with a needle and then connected to the monitor, which is worn against the body.
Abbott and Dexcom soon followed with devices presenting glucose data in real time. The accuracy of such meters has consistently improved over the past 20 years, and it is thanks to those advances that an artificial pancreas has become possible.
The ultimate goal is to replicate the entire job of the pancreatic control system, so that patients will no longer have to minister to themselves. But mimicking a healthy pancreas has proven exceptionally difficult.
Fundamentally, blood-glucose management is a problem in optimization, one that is complicated by meals, exercise, illness, and other external factors that can affect metabolism. In 1979, the basis for solving this problem was introduced by the biomedical engineers Richard Bergman and Claudio Cobelli, who described the human metabolic system as a series of equations. In practice, however, finding a solution is hard for three main reasons:
Insulin-action delay: In the body, insulin is secreted in the pancreas and shunted directly into the bloodstream. But when injected under the skin, even the fastest insulins take from 40 minutes to an hour to reach the peak of their action. So the controller of the artificial pancreas must plan on lowering blood glucose an hour from now—it must predict the future.
Inconsistency: Insulin action differs between people, and even within the same person at different times.
Sensor inaccuracy: Even the best continuous glucose monitors make mistakes, sometimes drifting in a certain direction—showing glucose levels that are either too low or too high, a problem that can last for hours.
The artificial pancreas reproduces the healthy body’s glucose-control system, which begins when carbohydrates are digested into glucose and ferried by the blood to the pancreas, which senses the increased glucose concentration and secretes just enough insulin to enable the body’s cells to absorb the glucose.
Two control systems based in the pancreas cooperate to keep blood-glucose concentrations within healthy bounds. One uses insulin to lower high levels of glucose, the other uses another hormone, called glucagon, to raise low levels. Today’s artificial pancreas relies on insulin alone, but two-hormone systems are being studied. Chris Philpot
What’s more, the system must take into account complex external influences so that it works just as well for a middle-aged man sitting at a desk all day as for a teenager on a snowboard, rocketing down a mountainside.
To overcome these problems, researchers have proposed various solutions. The first attempt was a straightforward proportional-integral-derivative (PID) controller in which insulin is delivered proportionally to the increase of blood-glucose levels and their rate of change. This method is still used by one commercial system, from Medtronic, after many improvements of the algorithm that adjusts the reaction of the PID to the pace of subcutaneous insulin transport. A more sophisticated approach is the predictive control algorithm, which uses a model of the human metabolic system, such as the one proposed in 1979 by Bergman and Cobelli. The point is to predict future states and thereby partially compensate for the delayed diffusion of subcutaneous insulin into the bloodstream.
Yet another experimental controller uses two hormones—insulin, to lower blood-glucose levels, and glucagon, to raise it. In each of these approaches, modeling work went far to create the conceptual background for building an artificial pancreas. The next step was to actually build it.
To design a controller, you must have a way of testing it, for which biomedical engineering has typically relied on animal trials. But such testing is time consuming and costly. In 2007, our group at the University of Virginia proposed using computer-simulation experiments instead.
Together with our colleagues at the University of Padua, in Italy, we created a computer model of glucose-insulin dynamics that operated on 300 virtual subjects with type 1 diabetes. Our model described the interaction over time of glucose and insulin by means of differential equations representing the best available estimates of human physiology. The parameters of the equation differed from subject to subject. The complete array of all physiologically feasible parameter sets described the simulated population.
In January 2008, the U.S. Food and Drug Administration (FDA) made the unprecedented decision to accept our simulator as a substitute for animal trials in the preclinical testing of artificial pancreas controllers. The agency agreed that such in silico simulations were sufficient for regulatory approval of inpatient human trials. Suddenly, rapid and cost-effective algorithm development was a possibility. Only three months later, in April of 2008, we began using the controller we’d designed and tested in silico in real people with type 1 diabetes. The UVA/Padua simulator is now in use by engineers worldwide, and animal experiments for testing of new artificial pancreas algorithms have been abandoned.
Perhaps one day it will make sense to implant the artificial pancreas within the abdominal cavity, where the insulin can be fed directly into the bloodstream, for still faster action.
Meanwhile, funding was expanding for research on other aspects of the artificial pancreas. In 2006 the JDRF (formerly the Juvenile Diabetes Research Foundation) started work on a device at several centers in the U.S. and across Europe; in 2008 the U.S. National Institutes of Health launched a research initiative; and from 2010 to 2014, the European Union–funded [email protected] consortium was active. The global frenzy of rapid prototyping and testing bore fruit: The first outpatient studies took place from September 2011 through January 2012 at camps for diabetic children in Israel, Germany, and Slovenia, where children with type 1 diabetes were monitored overnight using a laptop-based artificial pancreas system.
Most of these early studies rated the artificial pancreas systems as better than manual insulin therapy in three ways. The patients spent more time within the target range for blood glucose, they had fewer instances of low blood glucose, and they had better control during sleep—a time when low blood glucose levels can be hard to detect and to manage. But these early trials all relied on laptop computers to run the algorithms. The next challenge was to make the systems mobile and wireless, so that they could be put to the test under real-life conditions.
Our team at UVA developed the first mobile system, the Diabetes Assistant, in 2011. It ran on an Android smartphone, had a graphical interface, and was capable of Web-based remote observation. First, we tested it on an outpatient basis in studies that lasted from a few days to 6 months. Next, we tried it on patients who were at high risk because they had suffered from frequent or severe bouts of low blood glucose. Finally we stress-tested the system in children with type 1 diabetes who were learning to ski at a 5-day camp.
In 2016, a pivotal trial ended for the first commercial hybrid system—the MiniMed 670G—which automatically controlled the continuous rate of insulin throughout the day but not the additional doses of insulin that were administered before a meal. The system was cleared by the FDA for clinical use in 2017. Other groups around the world were also testing such systems, with overwhelmingly good results. One 2018 meta-analysis of 40 studies, totaling 1,027 participants, found that patients stayed within their blood-glucose target range (70–180 mg/dL) about 15 percent more of the time while asleep and nearly 10 percent more overall, as compared to patients receiving standard treatment.
Our original machine’s third-generation descendant—based on Control-IQ technology and made by Tandem Diabetes Care in San Diego—underwent a six-month randomized trial in teenagers and adults with type 1 diabetes, ages 14 and up. We published the results in the New England Journal of Medicine in October 2019. The system uses a Dexcom G6 continuous glucose monitor—one that no longer requires calibration by finger-stick samples—an insulin pump from Tandem, and the control algorithm originally developed at UVA. The algorithm is built right in to the pump, which means the system does not require an external smartphone to handle the computing.
The Control-IQ software predicts the rise in glucose concentration, to above 162 milligrams per decaliter of blood, by calculating extra doses of insulin, called correction boluses. A correction can be administered every hour, as needed. This is in addition to the continuous infusion of insulin throughout the day, known as the basal rate, which is varied every 5 minutes, according to the person’s insulin needs.
Tandem Diabetes Care
The Minimed 770G artificial pancreas, a hybrid system, manages metabolic insulin dosages—–it modulates the basal rate but does not administer correction boluses. It is descended from the first such system approved for general use.
Control-IQ still requires some involvement from the user. Its hybrid control system asks the person to push a button saying “I am eating” and then enter the estimated amount of carbohydrates; the person can also push a button saying “I am exercising.” These interventions aren’t absolutely necessary, but they make the control better. Thus, we can say that today’s controllers can be used for full control, but they work better as hybrids.
The system has a dedicated safety module that either stops or slowly attenuates the flow of insulin whenever the system predicts low blood glucose. Also, it gradually increases insulin dosing overnight, avoiding the tendency toward morning highs and aiming for normalized glucose levels by 7 a.m.
The six-month trial tested Control-IQ against the standard treatment, in which the patient does all the work, using information from a glucose monitor to operate an insulin pump. Participants using Control-IQ spent 11 percent more time in the target blood-glucose range and cut in half—from 2.7 percent to 1.4 percent—the time spent below the low-glucose redline, which is 70 mg/dL. In December 2019, the FDA authorized the clinical use of Control-IQ for patients 14 and up, and our system thus became the first “interoperable automated insulin-dosing controller,” one that can connect to various insulin pumps and continuous glucose monitors. Patients can now customize their artificial pancreases.
1. Two that have been approved by the FDA:
a. Medtronic MiniMed 670G
b. Control-IQ from Tandem Diabetes Care
2. Beta Bionics
3. Bigfood Biomedical
5. DreaMed Diabetes
7. Inreda Diabetic
8. Eli Lilly and Ypsomed
10. Then there are the many, many DIY projects underway.
The FDA approval came almost 14 years to the day after the expert in that Maryland conference room stated that the problem was unsolvable. A month after the approval, Control-IQ was released to users of Tandem’s insulin pump as an online software upgrade. And in June 2020, following another successful clinical trial in children with type 1 diabetes between 6 and 13 years old, the FDA approved Control-IQ for ages 6 and up. Children can benefit from this technology more than any other age group because they are the least able to manage their own insulin dosages.
In April 2021, we published an analysis of 9,400 people using Control-IQ for one year, and this real-life data confirmed the results of the earlier trials. As of 1 September 2021, Control-IQ is used by over 270,000 people with diabetes in 21 countries. To date, these people have logged over 30 million days on this system.
One parent wrote Tandem about how eight weeks on the Control-IQ had drastically reduced his son’s average blood-glucose concentration. “I have waited and toiled 10 years for this moment to arrive,” he wrote. “Thank you.”
Progress toward better automatic control will be gradual; we anticipate a smooth transition from hybrid to full autonomy, when the patient never intervenes. Work is underway on using faster-acting insulins that are now in clinical trials. Perhaps one day it will make sense to implant the artificial pancreas within the abdominal cavity, where the insulin can be fed directly into the bloodstream, for still faster action.
What comes next? Well, what else seems impossible today?
This article appears in the December 2021 print issue as “Creating the Artificial Pancreas.”