Nightscout FDA presubmission

Contents:

Cover letter

E-Copy Statement

To the

Food and Drug Administration
Center for Devices and Radiological Health
Document Control Center - WO66-G609
10903 New Hampshire Avenue
Silver Spring, MD 20993-0002

The eCopy is an exact duplicate of the paper copy, except for the following differences:

  • for security, a wet, inked signature is only provided on the hard copy, and absent in the digital and online versions of this eCopy
  • An exact copy of this eCopy has been permanently archived in git at tag 1.0.0 and may be retrieved from the website.

Nightscout sponsorship

To whom it may concern, Dr. Stayce Beck, et al.

This premarket submission is to discuss open source projects and FDA oversight. Specifically, this is to discuss the Nightscout project, aka “CGM in the Cloud.”

The sponsor, collectively known as Nightscout contributors, may be contacted through one of the core developers:

Ben West
4521 17th St. Apt 5
San Francisco, CA 94114

Additionally, core contributors openly discuss administration of the project via an email list administered by Google groups:

We request a meeting to discuss the risks and safety Nightscout. Please review the following discussion of the device description, future plans, and questions. We are happy to schedule meetings as needed.

Nightscout

Nightscout is a suite of open source projects. A smartphone provides ubiquitous network connectivity to Dexcom’s wireless receiver. After a polling period the last reading from the Dexcom receiver is transmitted to a database on the Internet. A website renders near-real-time views of the records stored in that database. Additionally, the website offers a RESTful API that a pebble watch can use to display the last known alarm status, trend, and glucose level as reported by the Dexcom receiver.

Device Description

Nightscout project

The Nightscout project is actually a suite of several independent projects:

When assembled, the completed device is called a “Nightscout rig.” In addition to the raw source code for these applications, other community maintained resources exist to help people learn how to assemble their own rigs. These include groups, photos, shared documents, videos on a variety of social media, including a centralized community curated website for documentation as well as community maintained forum software.

cgm-remote-monitor

This is a web application which simulates the display of a Dexcom receiver. In addition to showing the last known glucose level, it displays when the reading was taken, and offers a way to pan several hours retrospectively.

Every 5 minutes, a node.js server polls a mongo database, emitting the last readings over the last two days to any listeners subscribed to the server’s sgv websocket event. The server also serves a combination of html, css, and javascript to simulate a near-real-time display of the Dexcom receiver.

The web display works on most modern web browsers.

android-uploader

android-uploader is an Android application implemented in java. The application starts when a Dexcom receiver is detected using the operating system’s usb management system. The application reads data from the serial port made available by Dexcom’s usb connection, and uploads the latest record to a specified data backend. The backend may either be a RESTful API or a mongo database, and is configured using a preferences panel in the application.

The android-uploader source code must be compiled and distributed as an APK before it can run on an Android smartphone. Once installed and configured to upload data to the preferred cloud “backend”, a USB OTG cable is used to connect the smartphone to the Dexcom receiver’s micro usb port. The Dexcom receiver is a device cleared by the FDA for continuously monitoring glucose levels sampled from interstitial fluid. The receiver is designed to store and display values transmitted by the Dexcom sensor. android-uploader uses the serial connection provided by this usb capability to exchange data with the Dexcom receiver. The behavior of the uploader has been designed to behave as Dexcom would expect any data management system to behave. There is no expected difference in Dexcom’s behavior when the uploader smartphone is attached or while our software is auditing the records on the Dexcom receiver.

cgm-pebble

cgm-pebble is a C and javascript watch face developed using PebbleSDK. The javascript code runs on a Smartphone maintaining bluetooth connectivity to the Pebble watch. The javascript code retrieves information from cgm-remote-monitor and sends the last reading to over bluetooth to the pebble watch. The C code runs on the watch, receiving messages over bluetooth from a smartphone, and rendering the date, time, value, and trend as reported by a running instance of cgm-remote-monitor.

Development

Development takes place using Github, from the Nightscout organization page: https://github.com/nightscout/. Modifications, upgrades, development, and issue tracking happen using the resources connected to assets shared by a community of people. Each and every change to the source code is tracked by git and discussed through a Github pull request. Upgrades are provided by providing git merge requests, often using the Github UI, by identifying the last commit hash in use, and a verified change controlled path to apply latest updates from trusted contributors.

Assembly and guides

The git repositories merely provide the source code, and a verified way of exchanging source code for these projects. In order to be used, the source code must be configured, compiled, deployed, and installed.

While each repository contains instructions on how to test and work with that repository, the Nightscout development guides, forums, youtube videos, pictures, twitter account, and Facebook group provide “educational” material on how people have combined and configured these disparate parts to assemble something resembling a “medical device.” The web guides also reside in a git repository, where improvements are proposed by the community, reviewed, and adopted in similar manner to the source code itself.

The guides explain how to configure and install each component, with warnings of “things that might go wrong” at each phase. When people experience issues following the guides or during use, they use social media to find people that have similar issues or ask for help. There are also recommendations, optimized for cost and predictability, on which service providers are available, as well as how to work with those service providers.

Proposed Use

Nightscout

Nightscout is intended to be used as part of a data management system. The system provides for a “glanceable” secondary display of the information originating from the Dexcom CGM. A website allows the display to be presented on any device which can display websites to duplicate the display of the Dexcom.

The best way to get an idea of how people use Nightscout is to peruse the public testimonials of use, or the Facebook group.

http://i.imgur.com/q9gWTAd.png http://i.imgur.com/y8V7SMW.png http://i.imgur.com/hopdq1X.png http://i.imgur.com/a1rRb3X.png

Communicating recent therapy

The website URL is typically shared with caregivers and interested parties. This allows multiple people to monitor a Dexcom user’s glucose levels from any Internet connection. Multiple redundant displays eliminates transcription error and raises the fidelity of communicating current therapy status.

Glanceable interface

Displays are duplicated in multiple redundant locations. This alleviates people from needing to physically locate and attend to the receiver. For example, in scenarios where no therapeutic action is required, but the glucose levels must be considered, the glanceable display eliminates interruptions to existing activities. The lowered burden enables people to be more persistently aware, and therefore respond to scenarios with treatment with greater ease and fidelity.

Reliance on preexisting work

The Nightscout project relies on commodity components, as well as the excellent work from the folks at Dexcom. The android software interacting with the Dexcom receiver attempts to faithfully transmit data from the receiver to a configured storage/data management service hosted on the Internet. The android software is agnostic of the data management service, and can be configured to work with several different data management service providers. While Dexcom has not sanctioned or approved the data management functions performed by Nightscout, the software has been designed to behave in the way that Dexcom expects all data management systems to behave. The data management protocol was obtained through analysis of Dexcom’s own data management software. An open analysis of the source code listings and comparisons of behavior reveals that the behavior of the Dexcom receiver is unaffected when this system is in use, and matches the intended behavior of Dexcom’s own data management software. The use or non-use of Nightscout has no observable difference in the Dexcom equipment or system, either while Dexcom is in use or after. Eg, we believe that Nightscout has no effect on Dexcom’s performance, quality, or safety. Dexcom became an FDA approved device in 2012 and the accuracy of the device is well tested by thousands of patients. Providing connectivity to manage communication of the Dexcom’s readings makes no changes to the accuracy. Nightscout does not modify the blood glucose readings and thus maintains the original data quality.

When Nightscout is in use, the community recommends that users maintain their normal therapy. Nightscout should not alter therapy plans or decisions. Many of the community members recommend falling back to baby monitors, phone, SMS text message, self monitored finger-sticks, and physically checking the Dexcom receiver as tools to augment therapy, even while Nightscout is in use. The guiding philosophy behind this advice is that technology is a tool for managing therapy; that people administer therapy, not technology. Nightscout is another tool using commonly available technology, like baby-monitors, to bring diabetes therapy, specifically communicating current status of therapy, more in line with the way the users of these tools feel is acceptable.

Uses of Nightscout

Nightscout is useful any time remote near-real-time monitoring of Dexcom readings are desirable. People with diabetes find it useful to keep mindfulness of glucose levels while biking or other activities requiring both hands. People with diabetes, or PWD find it useful to communicate fidelity of therapy, as well as find support from their teams of caregivers.

Due to the ease of use, parents have been able to co-ordinate with school Nurse to prevent or treat injuries which are otherwise common. In some cases, use of Nightscout has helped gain insight into how common these injuries are, and we believe that the community aggregation tool can be used to report these injuries to the FDA for increased oversight of Dexcom and Medtronic devices in the marketplace. The community has also received reports of some parents using Nightscout to co-ordinate sleep-overs or camp visits, and in some cases walks with Grandpa, many for the first time, that would not other wise happen. They all cite Nightscout’s remote telemetry in liberating these activities, in some cases with pictures indicating injuries staved off or critical rescue care co-ordinated.

Adult users have cited Nightscout in increasing discretion. A common complaint among users of type 1 diabetes medical equipment is that the mandated use of the equipment combined with the time it takes to use the equipment often presents the unknowing public with a rude experience. It often appears that a PWD is ignoring someone by favoring a phone or pager or just producing rude beeps. When Nightscout is in use, the requirement to touch one of these medical devices disappears, which allows incorporating mindfulness more often and in a variety of different ways into the every day work flow. As a result, fewer interruptions from physically touching the medical device increases discretion because social disruptions are also reduced.

Requirements

Nightscout uploader device

An uploader device is an Android smartphone capable of “USB OTG” capability. These are commonly available. WIFI only versions, known as “android mini-computers” or and “Android TV box” are also commonly available. The prices vary widely from vendor to vendor, and depending on the cell network carrier subsidies.

Without any help, the DIY version requires downloading the source code from the Internet. Google’s Android software development kit is required to configure and compile the source listings from the git repository. This process requires that users know, or learn how to, prepare their device for debugging, go through basic debugging steps in order to configure, compile, and deploy the software as binary android package, and then install and run the software on their own smartphone.

Previous work

A number of informal discussions have taken place between the FDA and the Nightscout contributors. The discussions have revealed a need to establish a framework for open source authors and FDA to work together in order to best protect and promote public safety. We seek the FDA’s guidance in finding similar frameworks from other regulated or life-critical areas such as defense, aviation, and automotive industries.

Overview of Product Development

Community based, social technical development

Nightscout begins

As an open source project, the entire source code came into existence when people affected by type 1 diabetes with access to the best and safest therapy options found themselves unable to obtain therapy without any adverse events. In order to help monitor, communicate, and understand therapy, a few individuals created a data management system using commodity equipment allowing them to easily monitor the CGM without requiring physical access to the CGM receiver. Spurred by the improved family relationships and finding therapy easier to track, communicate, and manage, more and more people have added small improvements or helped others to gain liberties on their own. Many of these individuals cite “keeping their own children safe” as reasons for beginning their involvement with the project.

As of July 1, 2014, a dozen or so like-minded individuals record all proposed changes in their own Github forks or Github branches dedicated to discussing improvements or changes to a code base that is in active use by several dozen individuals and families. After the community reviews and tests these proposals in a public audit called a “pull request,” one of the core contributors accepts the changes into the “master” branch. This process of tracking, recording, and auditing work is sometimes called gitflow.

After the “master” branch has updated with changes relevant to the community, specially crafted pull requests allow tracking the exact git deltas necessary to bring another repository up to date with the community accepted versions. When community members report bugs, this tracking system allows developers to reproduce and co-ordinate fixes, in some cases specifically tailored to members’ needs.

For example, in one instance, a several individuals outside the U.S. needed displays in mmol/L vs mg/dL. A group of interested members teamed up to work on special mmol/l versions. The member actually completed the required changes, sharing the needed deltas with the group. As a result, we were able to re-use these same git tracking methods to compare and issue updates specifically for these users needing mmol/l.

Open source methodology

The development of Nightscout as an open source project follows a predictable development pattern to identify issues, incorporate bug fixes, as well as develop new features. The model, as discussed by Gabriel Coleman in Coding Freedom relies heavily on an open review process to share and distribute improvements.

The Software Freedom Law Center, in Transparent medical devices, also outlines the need for greater transparency in the operation of these medical devices. We encourage the FDA to adopt regulations that are consistent with the protections provided by open source methods, including Linus’s Law to make all bugs shallow and accessible to those affected by them. In designing Nightscout, we try to adhere to the design principles outlined in Unix philosophy in order to ensure safe, predictable, and effective operation of the Nightscout rig.

Known issues

There are several proposed improvements and known issues. One key feature liberating people, and thus making them safer, is the ease of use that accompanies data being made accessible to other trusted individuals. While we will adopt optional controls for authorizing and accessing data, parents of this system value easily sharing data with a school nurse with minimum hassle; and adults using this system value easily sharing their data as well.

Future plans

The sponsors would like to discuss appropriate regulatory controls that protect open source authors’ free speech as well as provides FDA with an appropriate framework to fulfill their mission. The sponsors passionately share the FDA’s mission to protect and promote public safety, a key reason Nightscout development is done as a “public performance,” and freely shared.

Oversight

Given the community’s frustration with safety in available medical devices to manage type 1 diabetes therapy, we believe there are opportunities for open source authors and FDA to work together. One such opportunity is in post-market surveillance. We have developed an aggregation tool which redisplays, depersonalized, many Nightscout remote monitors in a single “spaghetti plot.” We propose modifying the aggregating tool to automatically compile and submit reports to the FDA in order to aide in post market surveillance of devices used in diabetes therapy, and to generate useful research data on larger populations.

Integration

In the interest of safety, we need a single display to contextually manage type 1 diabetes. We will add data transfer from Medtronic insulin pumps to obtain “treatment” data consisting of the bolus wizard and bolus records. Additionally, the display will automatically show both the treatment data, carbohydrates consumed, insulin, and carbohydrate ratio, from insulin pumps overlaid with glucose readings from the Dexcom CGM.

In addition, we will also explore integrating with many other health, fitness, and nutrition APIs.

We will follow up with additional premarket submissions as required to discuss further development efforts. See also, decoding-carelink, and the pending decoding-carelink pre-submission.

Operational metadata

We anticipate adding indicators showing the connectivity status of the uploader device, as well as battery status, and other operational details of the system. These details will help quickly assess validity of the data, and whether or not the system is working and trustworthy.

Access controls

During development, the community has expressed an interest in developing access controls to help protect who can access displays. We anticipate development of “named views” which can be used to control who accesses the remote monitor website, as well as when and how. These views may optionally be protected by user name and password type of login system, or through creating unique and opaquely encoded tokens explicitly for sharing. We have found that the flexibility in sharing information in public outweighs the relative risks in the data being made public. As the community and software matures, we anticipate personalizing the access controls to meet the needs of its users.

Support/Commercialization

One criticism of open source is the lack of commercial support for individuals who lack the ability to safely assemble and operate their own rig. While the open source culture provides a large community able to train and offer support, the project remains accessible only to those with sufficient technical ability to assemble and debug their own equipment. We propose that the community would be safer if the public could buy assembled rigs on the market with support contracts to help ensure high quality operation for individuals lacking the time and effort. However, we are concerned that the current regulations considering this a “high risk” device prevents unprepared individuals from obtaining the help they need.

Specific Questions

  • Is this a secondary or passive or active display?
  • What does distributing mean?
  • What is the medical device? There are 6 separate open source projects, including this eCopy and the forums, all with separate maintainers and contributors. An assembled rig without the “cloud services” up and running does nothing.
  • If listing source code on the Internet is widely considered “free speech” how does this relate to “distributing free speech?”
  • Is there an API to upload data, relating to injuries or just for all therapy, to the FDA in order to aide in post-market surveillance?
    • Is there a way to prepare some kind of surveillance report, or observations of things we have found to the FDA outside of formal PMA/510k?
      • Should we build a “report to FDA” button into our UI to aide surveillance?
  • Should we develop a risk assessment framework for Nightscout?
  • Can we work on some framework to provide FDA with oversight for open source projects like Nightscout.
  • How would this project be categorized, pending the current MDDS guidance
    • Is this a real-time or “active” monitoring application?
      • Does classifying this as Class III, “high risk” medical device make sense, empower the public to pursue safety in their own therapy?
    • How can open source methodology be integrated into FDA controls?
      • We want FDA to enforce protections allowing owners of devices to collect the data from those devices. We might propose some kind of explore “time in range” endpoint which indicates absence of harm due to therapy? Can we nullify the proprietary protections given to vendors if they have not provided satisfactory evidence that their device enables safe therapy?
      • Is there a De Novo classification process that would make sense, given the relative risk?
  • Do any of the FDA certifications/approvals offer additional protection from liability, perhaps as outlined by Karen Sandler in “Killed by Code, Transparent Medical Devices?”

Feedback

The Nightscout contributors look forward to meeting and discussing these matters in depth. We will provide individuals to meet with, although as an open source project, no individual fully represents the community. We look forward to face to face conversations, emails, and phone calls per FDA availability. As members of the #WeAreNotWaiting movement, we encourage swift and iterative responses, integrated with the MDDS guidance, and the best regulatory science available.

Meeting Minutes

What follows are minutes of the first Nightscout FDA Pre-submission Meeting which took place on October 8, 2014.

Summary

  • FDA expressed interest in a continuing conversation related to the NightScout project
  • While Medical Device regulations are applicable to the NightScout project, Medical device requirements are implemented to ensure patient safety and were written to be flexible to address all types of medical devices and device manufacturers. How these regulations can be met was discussed with the agnecy.
  • Areas to be discussed – is there one responsible party? -
  • How are design controls met during software development?
  • How to fulfill post market requirements?
  • What product claims are made?
  • How are data security and privacy handled?
  • FDA and NightScout project members discussed performing a gap analysis regarding the regulations for the next discussion. Meeting time TBD.

Attendees

Participant introductions of attendees:

CGM in the Cloud / Nightscout

John Costik, Ping Fang, Ben West, and Scott Leibrand

Your Diabetes May Vary

Bennet Dunlap

Invited guests

Medtronic: Mark O’Donnell

FDA

Beth Stephen, Don St. Pierre, Robert Sauer, Stayce Beck, Courtney Lias, Katherine Serrano, Suzanne Schwartz, and Ariel Seeley

Discussion Topics

Introduction: FDA requested some background discussion on Nightscout’s software and project, history, open-source development methodology, and some existing controls, including general project oversight, how system updates are handled, and directions available to users.

FDA provided some initial comments to emphasize that FDA doesn’t have a bias against the open source route, nor is Nightscout the first time FDA has seen open source software. FDA emphasized that a key tenet of device regulation is that some entity be responsible for assuring the safety and effectiveness of the device and that postmarket safety is adequately monitored and that any potential problems are adequately addressed. FDA further emphasized that the requirements for medical devices were implemented to ensure patient safety and were written to be flexible to address all types of medical devices and device makers, and that FDA believes it is acheivable for Nightscout to meet applicable requirements even with non-traditional methods of development.

FDA asked questions about Nightscout’s current processes and how/whether they currently control software versions that go out to general use of Nightscout. Further questions were discussed about how Nightscout handles topics that should be addressed in device development, including quality, safety, and responsibility (e.g. Design Controls, Change Controls, Postmarket Surveillance, Claims, and Data/Cybersecurity) were also discussed. Nightscout provided a description of how core developers maintain the main community version, and review and test new features before they are incoportated or distributed to end users. Nightscout used an example of a current issue of incorrect time values to address questions such as:

  • How quickly are changes/problems addressed?
    • Determined by interest of developers and safety concerns
  • How is information on updates distributed?
    • Currently an informal process for development updates and checking
  • Who is responsible for code updates?
    • Ben currently triages, is the source code curator, coordinates efforts between developers. This activity is currently voluntary and informal.
  • How are updates tracked?
    • Website can track which version people are using, can push update requests to users.

The group discussed the FDA’s and and Nightscout’s shared goal of safety. The FDA hears the demand from patients for devices like Nightscout to exist, and doesn’t have a particular problem with safe and effective devices of this kind coming to market. In fact, they want to make this happen faster.

The FDA also recognizes the need for device interoperability to improve data access, along the lines of what Tidepool is working on. They noted recent progress on interoperability from CGM makers. Subsequent to the meeting, the FDA would like to hear from our community to help make sure interoperability moves forward.

The group discussed the FDA’s desire that there be a single responsible party to ensure that nothing falls through the cracks. The FDA emphasized the need for one party to take responsibility. For example, change

The group discussed FDA’s desire that there be a single responsible party to ensure that nothing falls through the cracks. The FDA emphasized the need for one party to take responsibility. For example, change control to determine when updates were ready to be implemented to general users, what problems need to be addressed and on what timeline, etc. should be adequately addressed. The FDA also asked about how problems are identified, reported, and addressed. They want to make sure there is a mechanism to handle, triage, and prioritize the response to complaints that might affect safety, to ensure that fixes can be distributed to people using the system, and to make sure that Nightscout is preventing unexpected complications from modifications. Nightscout stated that currently, how quickly a problem is addressed is determined by how quickly a developer picks up the issue and propose a solution. This may depend on the interest of the developers, the safety concern raised, and the complexity of the issue and its solution. The FDA discussed the benefits of implementing a process for evaluating the severity of issues and tailoring the speed at which solutions are developed to address the relative risk associated with the particular scenario. The FDA recognizes that Nightscout’s existing methods and processes, while informal, appear to address some of these concerns, but would like to see further documentation of how Nightscout is doing so, and wants to ensure that there is someone ultimately responsible. FDA interests include what happens if developers do not “jump” in quickly enough to develop solutions, the need to track issue severity, and concerns regarding the importance of reporting problems. Nightscout noted that they had determined that one person needed to be assigned as the lead. Ben West was voted by the core contributors to be the source code curator. FDA noted that was encouraging and that a similar approach could be more broadly implemented to address some of the FDA’s safety and regulatory concerns.

The group discussed how Nightscout might be able to provide the FDA with better visibility into any events that affect patient safety. The FDA would welcome submissions from Nightscout to MedWatch, especially if they can include as much detail as possible to see what was really going on. Such processes will help Nightscout make decisions on how to improve the system.

FDA requested that Nightscout begin working on a gap analysis to document what is already being covered, and which areas require improvements to come into compliance. The FDA requested a follow-up meeting with Nightscout within a few months.

About

In open source fashion, this document’s raw source, pdf version and an html rendering is available online. The purpose is to create a framework for having a discussion with the FDA.

Indices and tables