Archive for the ‘Flywheel’ Category

UHP Upgrade Update (Continued)

November 8th, 2020

1. Introduction

Good news — after a lot of hard work over the weekend by the CNI staff, we are most of the way back to normal operation with our upgrade to GE’s RX28 release.  The significant accomplishments over the weekend include solving the issues with the research SMS sequences as well as the spiral fieldmap sequence.  These are all operational again and the new sequences are fully integrated with Flywheel.

There are a few minor changes to the user interface with this revision of the scanner operating system and we describe these below.

2. Coil Selection

The tab for receiver coil selection has changed in RX28. Instead of a list of connected coils and checkmarks for indicating which coils are selected to be used, the new interface shows each port of the coil connection and highlights which ports are active. By default the Nova 32ch Head coil will have both Port P1 and Port P2 selected (as shown in the screenshot below).

3. Product Hyperband Updates

The Hyperband sequences on RX28 now have built-in calibration scans. The calibration can be integrated into the prescan of each Hyperband scan, so there is no need to do a separate ASSET Calibration before the Hyperband scans. The integrated calibration scan is turned on by default (as shown in the screenshot). If it is turned off, a separate calibration scan (named “Cal: [original series description]“) will be created automatically once the Scan button is pressed.  The Hyperband BOLD sequence also no longer requires an even number of slices / band to be acquired so it will improve the efficiency of some prescriptions over that in RX27.  In addition, the slice timing is correctly calculated and stored in the DICOMs. We will work with the developer of the dcm2niix utility to extract these timings and save them to the accompanying JSON file – a fix was needed because the developer had disabled this feature for GE Hyperband DICOMs given the erroneous timing values that were present for versions earlier than RX28.

RX28 UI Hyperband

4. High-Order Shim

GE has switched from a spiral-based acquisition for their high-order shim (HOS) process to a 3D fast gradient echo acquisition.  Users will need to replace their existing HOS series in their protocols with the new HOS template.  This can be accessed by first choosing Add Task, then selecting the GE protocol library, choose Adult->Other in the lefthand tabs and then select the hosGE entry.  We suggest using the GE HOS FOV28 template as this will most likely include all of your subject’s brain while still providing reasonable resolution. This new 3D FGRE approach may have modest benefits when subjects have significant B0 inhomogeneity due to orthodontics or large sinuses.

5. Remaining Tasks

As mentioned in our previous post, we will try to get the new coil configuration format for the NOVA16 research coil next week.  We are also working with Gary Glover on getting his spiral in/out (sprlio) pulse sequence installed.  The “show” system in the corner of the CNI scan room near the large TV screen is still running RX27 but we hope to update to RX28 in the next couple of weeks.

Thanks for your patience as we worked through this upgrade. We expect future s/w upgrades to be simpler now that the UHP is more closely integrated into GE’s standard product software.  Please don’t hesitate to contact us if you have any questions when using the new system.

Thanks and stay safe!

The CNI Team 

Update on Physiological Data Association in Flywheel

October 23rd, 2020

Dear CNI Users,

In August we wrote to you regarding a problem we found within the CNI Flywheel instance (cni.flywheel.io). In some cases the physiological data (respiratory and cardiac waveforms) from the scanner were not matched to the correct acquisition in Flywheel, or were not uploaded at all. We have determined that such failures occurred most often when a scan ended prior to the prescribed acquisition duration, The matching logic within the reaper used the duration information.  In some cases the mis-match led the reaper to associate the wrong physiological data with an acquisition; in other cases no match was found and thus no physiological data were uploaded.

We maintain backups of the physiological data, and this is enabling us to correct the association of the physiological data with its proper acquisitions. The restoration has been ongoing in the background for some time, and we expect to complete the process within the next couple of weeks.

Incorrectly associated physiological data in an acquisition will remain within the acquisition and be renamed using the following convention:

 <exam>_<series>_<acquisition>_original.gephysio.zip

Projects in which we found one or more acquisitions containing incorrect or missing physiological data will have a CSV file attached to the project. The files will be named using the following convention:

physio_fix_<group_name>-<project_name>.csv

Those ‘fix’ files will contain detailed information describing each acquisition that had physiological data associated with it. The contents of the file are described at the end of this message.

Please note that physiological data is only automatically associated for EPI-type sequences. If you store physiological data for other sequence (e.g. T2 spin-echo, T1-weighted fast gradient-echo, etc.) then you will need to manually upload the physiological data to Flywheel if you’d like it kept with your acquisition.  Please contact us directly if you have questions about this process.

We appreciate your patience as we work through this issue. If you have any questions regarding this process, or concerns about the data that have been restored to your sessions, please reach out and let us know either via email or Slack.

Contents of the ‘fix’ file

The ‘fix’ file is a text description (CSV) about the corrected physiological data.  It includes:

  • Session ID – A string representing the Flywheel Session ID
  • Acquisition ID – A string representing the Flywheel Acquisition ID
  • Flywheel Path - A human readable string showing the path to the acquisition in Flywheel
  • FIX Status – A string indicating whether the new data were ‘found’ or ‘replaced’
  • Message – describing the action performed, or an errors during execution.
  • Exam – Exam Number
  • Series – Series Number
  • TR – A number defining the TR of the acquired data in milliseconds
  • NTP – An integer defining the  number of temporal samples in the physiological data file
  • RAW NTP Number of temporal positions (from the DICOM or PFile header),
  • Scan Duration – Calculated scan duration
  • PPG_Samples - the number of Physio Samples
  • PPG_Lines_Expected – the number of lines we expected in the PPG file
  • PPG_Lines – the number in samples found in the PPG file found in Flywheel (if applicable)
  • PPG_Lines_Fix – Number of sample found in the Matched/Fixed data,
  • DELTA_FIX_EX – the difference between the fixed data and what was expected
  • DELTA_FW_EX – the difference between what was in Flywheel and what we expected
  • Date and Time – of the acquisition
  • FIX Time Delta Difference between expected timestamp and FIX file timestamp in Sec
  • FW Time Delta Difference between expected timestamp and FW file timestamp in Sec
  • FW File Base Base file name of data found in Flywheel
  • Regx – The regular expression used to find the matched data

Thanks and stay safe!

The CNI Team

Flywheel: Part Four

August 29th, 2018

Flywheel

We are on track to switch over to Flywheel on Saturday, September 1.  As of that day, new data collected at CNI will not be placed in NIMS.  New data will be sent  to Flywheel directly (https://cni.flywheel.io).  NIMS will be kept online as legacy only.

Logins for the Flywheel site is managed by Stanford authentication, via Google. So to login click the ‘Sign in with Google’ button.

 

Login page of the Flywheel data management system

Login page of the Flywheel data management system

 

Freezing NIMS

Users often make changes to NIMS after data are uploaded, so we will permit edits (e.g., notes) until Saturday, Sept. 8th.  However, any edits made after August 31 will not be automatically carried over to Flywheel (which we have been doing).  After Sept. 8th we plan to freeze the NIMS database and make it read-only.

Data from October 1 2018 are already in the new Flywheel system.  NIMS contains about 7 years worth of legacy data.  Over the next year we plan to migrate all the NIMS data into Flywheel as well. If you would like specific Projects in NIMS integrated sooner, please let us know.

NIMS-FS

People who use NIMS-FS (through the CNI Linux Containers) should note that no new data will be accessible through that mechanism. Those wishing for “command line access” to CNI data will need to use a different technology.  The Flywheel CLI (command line interface) is an option for those of you who have used the features in NIMS-FS.  The CLI has methods for importing and exporting data from your authorized projects, and it runs on multiple platforms.  The CLI can be downloaded from your profile page within Flywheel. If you’re interested, please contact Michael Perry for instructions about how to install and use the Flywheel CLI.

- The CNI Team

 

Flywheel: Part Three

August 6th, 2018

Thanks to all of you who have started to take a look at the Flywheel data management system. A few notes and reminders.

1.  We are still on track for a September 1 change over.  Having said this, Michael is considering transitioning a little bit sooner.  The reason is that the very heavy use of the scanner this month is stressing the limits of our NIMS storage.  But we may be able to make it to September 1st with both NIMS and Flywheel running.

2. Starting this week, Michael will join Laima for a portion of the new user training to explain the Flywheel interface .  Knowledge is viral around this place – so check with new users, Michael, and Laima if you have questions.  And pass on what you learn.  There is also a plan to step up the documentation by mid-September.

3.  A number of people have asked us about the Flywheel technology.  Here are a few principles.

Cloud. The Flywheel system (https://cni.flywheel.io) is on the Google Cloud Platform.  This is an advantage to the CNI because we no longer have to maintain computers and back up the data.  In the cloud we can leverage (and pay for) more computing power for a short time if there is a big or rush job.  So far, our experience with pricing is good.

Data storage. The CNI has more than seven years of data including more than a 100,000 acquisitions.  The size of the data has increased dramatically with the new SMS sequences that many of you use (ask me why when you take my class). We have always worried about the cost of storing the data.  The largest files (e.g., the P-files) comprise almost 80% of our storage and these data are rarely used.  The Flywheel system uses ‘object storage’, which is the least expensive type of storage. Rarely accessed files drift off into ‘cold storage’, which is particularly inexpensive. This should prove to be cost effective for us while still allowing us to reconstruct data that was collected even a few years ago.  Which we have done for some users!

Computation.  The computations performed by the Center Edition run in  a simple framework that Flywheel calls a Utility Gear; this is a Docker Container coupled with a format for setting the program parameters.  Flywheel jobs, such as reconstructing SMS scans, or converting a file from DICOM to NIfTI, are carried out by Utility Gears.  In the cloud, we can invoke multiple Gears at once on many machines that are  provided temporarily (cloud scaling).  We pay for the machines only for the few minutes or hours they are in use.  Cloud-scaling is useful when we have to run a lot of jobs – such as re-running multiple recons to fix an error or improve an analysis. This functionality provides us with greater scalability and performance than we could provide through the computers we purchased with grant funding.

4. To install Flywheel, the CNI team had to keep NIMS running and develop a system to convert seven years of prior data; Michael has done a great job with those tasks. CNI is not the first site to run Flywheel on the Google Cloud because those sites had no legacy storage.  When we fully switch over, the CNI installation will be the largest by far, and we are determined to be the best! Go Cardinal!

 

NIMS transition to Flywheel: Part Two

July 17th, 2018

Main point

The Flywheel data management system has been running smoothly for several months at the CNI.  The main purpose of this message is to encourage you to explore Flywheel (https://cni.flywheel.io). A number of groups have already switched from NIMS to take advantage of Flywheel’s features. After Michael completes further testing, which we anticipate to be around September 1, we will stop sending new MRI data to NIMS and instead send data directly to Flywheel (see note). So this is a good time to learn about the new system.

During the Fall we will hold informal sessions at the CNI to explain the system and its features. We will also write more blog posts to explain the system and what motivated us to make the transition.

About Flywheel

Flywheel offers two levels of service, the Center and Lab Editions; the CNI runs the Center Edition. This version does all  the things that NIMS does, including automatically uploading data (e.g., P-files (raw data), DICOM data, and Physio data) and converting data to NIfTI format, which is what most users download.

The Center edition also has features that exceed what NIMS could do. Below is a partial list of Flywheel features that are relevant to CNI users.

  • Session Templates – For each project you can set up rules that automatically check that the expected data were collected, and flag sessions with missing data
  • Notes - Add notes about a session or acquisition (e.g., subject moved; life is good; buy milk)
  • Elastic Search – Find specific types of data, notes, and more
  • Visualization – Flywheel includes viewers for many types of data (NIfTI, DICOM, EEG, OBJ,  CSV, etc.)
  • Collections – Gather data from multiple projects to form a Collection of data
  • Permissions – Adding and removing access to users is simpler, and there are different permission levels available
  • Upload – You can add behavioral data, say from REDCap or or your own notes, to an existing session
  • Gears – Users can run one of a set of Utility Gears (Analysis Gears are available in the Lab Edition)
  • Gear Rules – Users can determine which set of Utility Gears are run on their data. For example, want to use dcm2niix for your DICOM data? Simply setup a Gear Rule to execute dcm2niix for all DICOM data coming in to a given project (ask Michael to show you how, or have a look at the doc).

There are many other features we think CNI users will benefit from. Please explore the interface and browse the documentation available in the UI.

Finally, please reach out to Michael Perry (lmperry@stanford.edu) if you have any questions or experience issues with the platform. In this early phase we expect things to come up and we welcome the opportunity to help navigate through this transition.

Note:  During this testing phase there is a delay before the data appear in Flywheel. Most data are uploaded first to NIMS, where we start most reconstructions and data processing. When testing is completed, in September, Flywheel will handle all reconstructions and data processing while NIMS will switch over to an archival mode.

The CNI Team

NIMS Transition to Flywheel: Part One

February 9th, 2018

BACKGROUND

When planning the CNI we committed to providing the community with data management services. As many of you know, most MRI centers do little more than hand the user a DVD at the end of the session, and wish them well. CNI users are supported much more extensively. Data acquired on our scanner are immediately transferred to the Neurobiological Image Management System (NIMS). Once in NIMS, the data can be reconstructed and converted into the formats that most of our community uses (e.g., NIfTI). These are the data that most users download and use in their research.

Over the years the CNI community has accumulated a great deal of data. NIMS contains the work of about 900 researchers. These data comprise more than 15,000 sessions from more than 6,000 subjects. There are more than 45,000 fMRI scans, nearly 100,000 anatomical scans, around 5000 diffusion scans and over 500 spectroscopy scans.

The CNI remains committed to supporting the research of its user community by providing state-of-the art data management. To this end we are excited about transitioning from NIMS, our home-grown data management system, to Flywheel.

THE ISSUES

Over the years there have been a series of updates to NIMS to accommodate the growing data set. In addition to updates to the NIMS software itself there have been several hardware upgrades over the last few years, including both increased storage and increased computational power; we were fortunate receive grants to support improvements from the Neurosciences Institute. At present NIMS is comprised of a 200TB main file server and a 200TB off-site backup file server, three compute servers (with more than 80 cores, 2TB of RAM, and 14TB of SSD scratch), and a powerful web server, all interconnected by a 10 Gigabit network. The NIMS hardware and associated software are maintained by Michael and the CNI staff.

The support, both physical and financial, of this system is a burden on the CNI. Some of you have may have experienced the limitations of NIMS with regard to data reconstructions, or the limited feature set around search, sharing, and permissions, or downloading bulk datasets, or other missing features that fall into the “Wouldn’t it be nice if…” category.

THE SOLUTION

Over the last few years a number of us (Wandell, Schaefer, Dougherty, Perry, and others) were supported by the Simons Foundation to design the next generation of NIMS. This project (The Project on Scientific Transparency) has also been supported by and integrated with the work being done by Russ Poldrack and Chris Gorgolewski (OpenfMRI) – supported by the Arnold Foundation. This joint effort culminated in the development of an open-source framework to support the next generation of data management – SciTran.

Flywheel has adopted SciTran and engineered a modern user interface and infrastructure that makes it easy to use, maintain, and scale. Flywheel has a multitude of new features over NIMS; runs on Google Cloud Platform; and has a sustainable support strategy.

We will be describing Flywheel more fully in the coming weeks, first reaching out to lab managers and individual groups with the goal of rolling it out to the user community broadly in the coming months. We look forward to working with all of you to make the transition smooth and to help you take advantage of new opportunities that are enabled by the next generation system.

 

See you around the magnet!
- The CNI Team