The Virtual Brain (TVB) is a large-scale brain simulator. With a community of thousands of users around the world, TVB has become a validated, popular and standard choice for the simulation of whole brain activity. TVB users can create simulations using neural mass models which can produce outputs for different analysis and modalities. TVB allows scientists to explore and analyze both simulated and experimental data. It contains analytic tools for evaluating relevant scientific parameters in light of that data. The current implementation of TVB is written in Python, with limited large-scale parallelization over different parameters. The objective of TVB-HPC is to enable large-scale parallelization of TVB simulations by making use of high performance computing to explore large parameter spaces for the models. With this approach, neuroscientists can define their models in a domain specific language based on NeuroML and automatically generate code which can run either on GPUs or on CPUs with different architectures and optimizations. The result is a framework that hides the complexity of writing robust parallel code and offers neuroscientists a fast and efficient access to high performance computing. TVB-HPC is publicly available on GitHub and, at the end of HBP project phase SGA2, it will be possible to launch large parameter simulations using code automatically generated with this framework via the HBP Collaboratory.
Arbor is a software library designed from the ground up for simulators of large networks of multi-compartment neurons on hybrid/accelerated/many core computer architectures.
Performance portability was completed for the three main target HPC architectures available through the HBP: Intel x86 CPUs (AVX2 and AVX512), Intel KNL (AVX512) and NVIDIA GPUs (CUDA).
Optimized kernels are automatically generated to target each architecture, and the system used in Arbor can be extended to new architectures in the future.
The other enhancements and features implemented in Arbor are:
Fully parallelized event generation and queuing from spikes.
Efficient sampling of model state on CPU and GPU implementations, e.g. voltage and current.
Significant refactoring to prepare the code for general release.
A Python interface for users.
The source code was released publicly on GitHub with an open source BSD license, along with documentation on Read the Docs, and automatic testing was set up on Travis CI.
The development of VTK-m was co-funded by the HBP during the second project phase (SGA1). This page is kept for reference but will no longer be updated.
VTK-m is a scientific visualization and analysis framework that offers a wealth of building blocks to create visualization and analysis applications. VTK-m facilitates scaling those applications to massively parallel shared memory systems, and it will – due to its architecture – most likely also run efficiently on future platforms.
HPX is a task-based programming model. It simplifies the formulation of well-scaling, highly-parallel algorithms. Integrating this programming model into VTK-m streamlines the formulation of its parallel building blocks and thus makes their deployment on present and emerging HPC platforms more efficient. Since neuroscientific applications require more and more compute power as well as memory, harnessing the available resources will become a challenge in itself. By combining VTK-m and HPX into task-based analysis and visualization, we expect to provide suitable tools to effectively face this challenge and facilitate building sophisticated interactive visual analysis tools, tailored to the neuroscientists’ needs.
Parallel primitive algorithms required for VTK-m have been added to HPX along with API support to enable the full range of visualization algorithms developed for VTK-m. A new scheduler has been developed that accepts core/numa placement hints from the programmer such that cache reuse can be maximized and traffic between sockets minimized. High performance tasks that access data shared by application and visualization can use this capability to improve performance. The thread pool management was improved to allow visualization tasks, communication tasks, and application tasks to execute on different cores if necessary, which reduces latency between components and improves the overall throughput of the distributed application. RDMA primitives have been added to the HPX messaging layer. These improvements make it possible to scale HPX applications to very high node/core counts. Respective tests have been successful on 10k nodes using 650k cores.
The development of PLIViewer was co-funded by the HBP during the second project phase (SGA1). This page is kept for reference but will no longer be updated.
The PLIViewer is visualization software for 3D-Polarized Light Imaging (3D-PLI), to interactively explore the scalar and vector datasets; it provides additional methods to transform data, thus revealing new insights that are not available in the raw representations. The high resolution provided by 3D-PLI produces massive, terabyte-scale datasets, which makes visualization challenging.
The PLIViewer tackles this problem by providing functionality to select areas of interests from the dataset, and options for downscaling. It makes it possible to interactively compute and visualize Orientation Distribution Functions (ODFs) and polar plots from the vector field, which reveal mesoscopic and macroscopic scale information from the microscopic dataset without significant loss of detail. The PLIViewer equips the neuroscientist with specialized visualization tools needed to explore 3D-PLI datasets through direct and interactive visualization of the data.
The original dataset: Fibre Orientation Maps rendered on top of Retardation mapA full slice from the Fibre Orientation Map of a Vervet MonkeyOrientation Distribution Functions (ODFs) rendered with Streamline TractographyClose-up view of the ODFs
The development of this technology was co-funded by the HBP during the second project phase (SGA1). This page is kept for reference but will no longer be updated.
Complementary to other viewers and visualization implementations for NEST simulations, this technology offers a rendering of activity and membrane potentials in a neural network simulated with NEST.
A prototypical implementation exists that is based on vtk, the widely-used visualisation toolkit. This implementation can be generally run by computational neuroscientists on their workstations, imposing only moderate hardware requirements. Experiments using rendering on high-performance computing infrastructure were successful.
The results indicate that this component is extensible towards large-scale simulations that require HPC resources and thus produce large output data. The hi-fidelity rendering used in this case provides very high quality images that may be suitable for publications (Proof of concept image below).
Rendering of color-coded membrane potentials on spatial neurons from a running NEST simulationProof of concept of a high-quality rendering of spatially organized point neurons
The Multi-View Framework is a software component, which offers functionality to combine various visual representations of one or more data sets in a coordinated fashion. Software components offering visualization capabilities can be included in such a network, as well as software components offering other functionality, such as statistical analysis. Multi-display scenarios can be addressed by the framework as coordination information can be distributed over network between view instances running on distributed machines.
The framework is composed of three libraries: nett, nett-python and nett-connect. nett implements a light-weight underlying messaging layer enabling the communication between views, whereas nett-python implements a python binding for nett, which enables the integration of python-based software components into a multi-view setup. nett-connect adds additional functionality to this basic communication layer, which enables non-experts to create multi-view setups according to their specific needs and workflows.
Interactive optimization of parameters for structural plasticity in neural network models (top left); comparative analysis of NEST simulations (top right); statistical analysis of NEST simulations (bottom left); multi-device and multi-user scenarios (bottom right)
Science has driven the development of the NEST simulator for the past 20 years. Originally created to simulate the propagation of synfire chains using single-processor workstations, we have pushed NEST’s capabilities continuously to address new scientific questions and computer architectures. Prominent examples include studies on spike-timing dependent plasticity in large simulations of cortical networks, the verification of mean-field models, models of Alzheimer’s and Parkinson’s disease and tinnitus. Recent developments include a significant reduction in memory requirements, as demonstrated by a record-breaking simulation of 1.86 billion neurons connected by 11.1 trillion synapses on the Japanese K supercomputer, paving the way for brain-scale simulations.
Running on everything from laptops to the world’s largest supercomputers, NEST is configured and controlled by high-level Python scripts, while harnessing the power of C++ under the hood. An extensive testsuite and systematic quality assurance ensure the reliability of NEST.
The development of NEST is driven by the demands of neuroscience and carried out in a collaborative fashion at many institutions around the world, coordinated by the non-profit member-based NEST Initiative. NEST is released under GNU Public License version 2 or later.
How NEST has been improved in HBP
Continuous dynamics
The continuous dynamics code in NEST enables simulations of rate- based model neurons in the event-based simulation scheme of the spiking simulator NEST. The technology was included and released with NEST 2.14.0.
Furthermore, additional rate-based models for the Co-Design Project “Visuo-Motor Integration” (CDP4) have been implemented and scheduled for NEST release 2.16.0.
NESTML is a domain-specific language that supports the specification of neuron models in a precise and concise syntax, based on the syntax of Python. Model equations can either be given as a simple string of mathematical notation or as an algorithm written in the built-in procedural language. The equations are analyzed by NESTML to compute an exact solution if possible, or use an appropriate numeric solver otherwise.
This technology couples the simulation software NEST and UG4 by means of the MUSIC library. NEST can only send spike trains where spiking occurs; UG4 receives those in form of events arriving at synapses (timestamps). The time course of the extracellular potential in a cube (representing a piece of tissue) is simulated based on the arriving spike data.The evolution of the membrane potential in space and time is described by the Xylouris-Wittum model.
Link to this release (2017): https://github.com/UG4
The Virtual Brain (TVB) is a large-scale brain simulator. With a community of thousands of users around the world, TVB has become a validated, popular and standard choice for the simulation of whole brain activity. TVB users can create simulations using neural mass models which can produce outputs for different analysis and modalities. TVB allows scientists to explore and analyze both simulated and experimental data. It contains analytic tools for evaluating relevant scientific parameters in light of that data. The current implementation of TVB is written in Python, with limited large-scale parallelization over different parameters. The objective of TVB-HPC is to enable large-scale parallelization of TVB simulations by making use of high performance computing to explore large parameter spaces for the models. With this approach, neuroscientists can define their models in a domain specific language based on NeuroML and automatically generate code which can run either on GPUs or on CPUs with different architectures and optimizations. The result is a framework that hides the complexity of writing robust parallel code and offers neuroscientists a fast and efficient access to high performance computing. TVB-HPC is publicly available on GitHub and, at the end of HBP project phase SGA2, it will be possible to launch large parameter simulations using code automatically generated with this framework via the HBP Collaboratory.
Arbor is a software library designed from the ground up for simulators of large networks of multi-compartment neurons on hybrid/accelerated/many core computer architectures.
Science has driven the development of the NEST simulator for the past 20 years. Originally created to simulate the propagation of synfire chains using single-processor workstations, we have pushed NEST’s capabilities continuously to address new scientific questions and computer architectures. Prominent examples include studies on spike-timing dependent plasticity in large simulations of cortical networks, the verification of mean-field models, models of Alzheimer’s and Parkinson’s disease and tinnitus. Recent developments include a significant reduction in memory requirements, as demonstrated by a record-breaking simulation of 1.86 billion neurons connected by 11.1 trillion synapses on the Japanese K supercomputer, paving the way for brain-scale simulations.