An efficient 3D extraction and reconstruction method for myelinated axons of mouse cerebral cortex based on mixed intelligence

: Accurate reconstruction of the 3D morphology and spatial distribution of myelinated axons in mouse brains is very important for understanding the mechanism and dynamic behavior of long-distance information transmission between brain regions. However, it is difficult to segment and reconstruct myelinated axons automatically due to two reasons: the amount of it is huge and the morphology of it is different between brain regions. Traditional artificial labeling methods usually require a large amount of manpower to label each myelinated axon slice by slice, which greatly hinders the development of the mouse brain connectome. In order to solve this problem and improve the reconstruction efficiency, this paper proposes an annotation generation method that takes the myelinated axon as prior knowledge, which can greatly reduce the manual labeling time while reaching the level of manual labeling. This method consists of three steps. Firstly, the 3D axis equation of myelinated axons is established by sparse axon artificial center point labels on slices, and the region to be segmented is pre-extracted according to the 3D axis. Subsequently, the U-Net network was trained by a small number of artificially labeled myelinated axons and was used for precise segmentation of output by the last step, so as to obtain accurate axon 2D morphology. Finally, based on the segmentation results, the high-precision 3D reconstruction of axons was performed by imaris software, and the spatial distribution of myelinated axons in the mouse brain was reconstructed. In this paper, the effectiveness of this method was verified on the dataset of high-resolution X-ray microtomography of the mouse cortex. Experiments show that this method can achieve an average MIoU 81.57, and the efficiency can be improved by more than 1400x compared with the manual labeling method.


INTRODUCTION
The brain is the most complex organ in the human body, responsible for perception, thinking, memory and action. Until now, there is still a long way to go to truly understand brain function and effectively treat brain diseases [1]. Previous studies of information transfer patterns in the brain focused on dendrites and synapses, but the spatial distribution of dendrites could only explain the local information transfer patterns in the brain [2]. Recent research has found that long-distance transmission of information across brain regions may be the key to understanding the patterns of information transmission in the brain. [3] compared the effects of the presence and absence of long-distance transmission modes on information transmission in whole brain model, and concluded that the presence of long-distance transmission not only improves the brain's sensitivity to information but also reduces the error of information transmission. By analyzing fMRI data, [4] theorized that individual a yan-fang@sjtu.edu.cn b jiejiren@sjtu.edu.cn c zfshao@sjtu.edu.cn d *XL3A@sjtu.edu.cn differences between people may come from long-distance transmission across brain regions; [5] found that people with autism lack long-distance functional connections between brain regions. Modern anatomy has proved that myelinated axons are the material basis for information transmission between brain regions [6]. However, the spatial distribution of myelinated axons in the whole brain at the microscopic level is still unclear, and structural analysis of the differences in functional connections above cannot be conducted.
Currently, the main techniques used to image the 3D microstructure of myelinated axons in the brain are microscopy based on fluorescence and electron microscopy based on conductive metal. However, the optical microscope has the visible light diffraction limit, so the resolution is not enough to analyze all myelinated axons. In addition, when using optical microscope (OM) for 3D imaging, physical section [7], [8], [9] or tissue transparency [10], [11], [12] of the sample is required; electron microscope (EM) can achieve the resolution of nanometer level, but field of view of EM is very small. If three-dimensional imaging is to be carried out, continuous sectioning [13], [14], [15] or ion beam cutting [16], [17], [18] is needed. In addition, the above two technologies are difficult to achieve three-dimensional isotropy, which directly hinders the quantification of imaging results. Three-dimensional X-ray imaging based on heavy metal staining has become the most powerful tool for structural imaging of myelinated axons, because it can achieve a much larger filed of view than electron microscopy at a resolution of up to 100 nanometers [19]. And it can achieve three-dimensional isotropy without intruding [20].
Recently, deep learning-based methods for the semantic segmentation of myelinated axons in images have reached a state of art level [21]. However, due to the large number of myelinated axons in the brain and the large differences in the spatial distribution of axons among different regions, a large number of manually annotated images are needed as the training set to obtain a reliable segmentation network. But a large number of manually annotated images will generate huge labor time costs. Therefore, it is necessary to reduce the workload of manual labeling.
In this paper, we propose a new data-set labeling method by taking full advantage of two structural features of myelinated axons in X-ray microtomography results to reduce the manual labeling workload:(1) myelinated axons show a continuous tubular structure in three dimensions; (2) The diameter variation range of the same medullated axon is small.
The method consists of three steps: Firstly, an axon axis model is established, which takes full advantage of the three-dimensional continuous tubular structure of medullated axons. The axis model of the myelinated axon is generated by artificially marking a few center points of a myelinated axon. Then, the area determined by the axis model was segmented, and the U-Net [22] based segmentation was performed on the images within a certain range near the axis model based on the characteristics that the diameter of the same axon at different positions was small and the conditions around a single axon were similar, to obtain high-precision segmentation results of this myelinated axon. Finally, the result is reconstructed slice by slice in three dimensions. The proposed method was quantitatively analyzed on the European light source dataset [19].

METHODS
The framework diagram of the proposed method is shown in Figure 1. There are three main steps: axis extraction of myelinated axons, segmentation area according to the axis, and reconstruction of the segmentation results. The first step is to determine the axis. With the help of machine learning, a small number of hand-marked central points were fitted to obtain the axis of myelinated axons. A square of fixed size centered on a fitting axis in two dimensions defines the area to be segmented. In the second step, the region around the axis is segmented based on U-Net [22]. A small number of two-dimensional intact myelinated axons were manually labeled and used as a training set to train the U-Net network. The results of the first step were input into the U-Net to obtain the twodimensional morpho-logical information of medullated axons. The third step is to reconstruct the three dimensions information of myelinated axons. The results obtained in the second step were reconstructed to obtain the spatial distribution of myelinated axons.

Axis Extraction of Myelinated Axon
Many existing methods train segmentation networks using manually labeled myelinated axons slice by slice in two dimensions as annotation sets [21]. However, most approaches ignore the three-dimensional continuity of myelinated axons, and treat each 2D image of myelinated axons as a separate object, resulting in a huge amount of labeling work. Therefore, in the first step of the proposed method, the axis is automatically extracted by a very small amount of marked central points taking full advantage of the three-dimensional spatial continuity of myelinated axons. The dimension of X-ray microscopic 3D imaging of mouse cerebral cortex is denoted as [x,y,z]∈N 3 , where the x-y plane is the tomographic plane of 3D reconstruction and z is the rotation axis direction of the sample projection. Based on the spatial structure characteristics of myelinated axons, a polynomial regression method was used to construct the axis model of myelinated axons with a small number of markers.
The coordinate set of the center point of the artificially marked part of the axon was denoted as D, which was used as the training set of the axon axis extraction model.
Where (x z , y z ) is the position of the center of a myelinated axon on the z layer; z is the z layer; m is the number of layers between the two marks. 1 ≤ m ≤ z max ; z max is the maximum value of z ; i is serial number.
Establish the n-order polynomial model with z as the independent variable:

{
x z = w x0 + w x1 z + w x2 z 2 + … + w xn z n ŷ z = w y0 + w y1 z + w y2 z 2 + … + w yn z n Where = × ⅈ, 0 ≤ ⅈ ≤ ( ⁄ ); (w x0 ,w x1 ,w x2 ,⋯,w xn ), (w y0 ,w y1 ,w y2 ,⋯,w yn ) are the model parameters. is the order of the model which depends on the root mean square error. The least square method is used to solve the parameter matrix of the axis model. When the model is determined, in the value into the model when m=1, get all (x z ,ŷ z ), namely the axis of this myelinated axon is determined. According to the characteristic that the diameter of a myelinated axon changes little, the raw images within a fixed range near the axis of a myelinated axon are extracted to obtain a set of two-dimensional images containing this myelinated axon.

Segmentation Based on U-Net
There are a large number of myelinated axons in the mouse brain, and the direction of different axons is different. Therefore, when a cuboid is used to cut out an image containing a whole three-dimensional myelinated axon from the original image, other myelinated axons must be included. At this time, if only labeling the selected axon and using it as a training set to train the segmentation model, a large error will be generated (Table 1), and if all myelinated axons in the selected image are to be labeled, more labor costs will be spent. Therefore, in the second step, axis extraction is carried out on the labeled images, and the original images and annotations within a fixed range near the axis of the myelinated axon are extracted according to the diameter of a myelinated axon has little change and the cortex near myelinated axons are similar. A set of images and annotations containing a single myelinated axon are obtained. Mark I z (x,y) as the label for one myelinated axon on the z layer. Calculate gradient ∇I z (x,y) for the annotation data, and determine the myelinated axon boundary point parallel to the edge of the image according to the non-zero points on the gradient map. Calculate the center point according to the boundary point, and calculate the axis of the myelinated axon marked after calculation for each layer.
The standard U-Net network was trained to obtain a model for segmentation near the axis of myelinated axons using the original images and annotations within a certain range near the axis. After segmenting the output of the first step, the holes are filled according to the spatial continuity of myelinated axons, and the isolated points are filtered out in three dimensions to obtain the final segmentation results. Fig 3: The architecture of U-Net. After the image is segmented by U-Net, the image is filled with holes and the isolated part in 3D is filtered to get the segmentation result.

Three-dimensional Reconstruction of Myelinated Axons
In order to obtain information of a single myelinated axon in three-dimensional space, after obtaining its twodimensional segmentation results, it is necessary to return the segmentation results according to the location of its axis. After the homing of each segmented myelinated axon, the three-dimensional spatial information of multiple myelinated axons in a stack was obtained.

Evaluation Dataset
We evaluated the proposed method by dataset of highresolution X-ray microtomography of mouse cortex. The structures labeled by heavy metals formed contrast in the image including the structures of various cell membranes and subcellular membrane structures in the cerebral cortex of mice. Myelinated axons were identified in this data set because their outer layers were covered with a large number of phospholipid membrane structures formed by oligodendrocytes. The annotations of myelinated axons were done by experienced neuroscience researchers. The pixel size of the original image is 30nm. We selected the image with the size of 512x512x800 pixel³ in the original image as the data to generate the annotation set, without any additional processing. Manual labeling produced four myelinated axons, one of which was used as the training set of U-Net; the remaining three are marked by only part of the center points, and the image of the axis near the axis of the first phase of the proposed method is used as the test set.

Implementation Details
In the first step, building the axis model of myelinated axons used the sklearn library of python.
In the second step, the U-Net network is deployed on a PyTorch-based library, trained and verified using the NVIDIA GeForce GTX 980 Ti. Standard data amplification methods (mirroring, flipping) were used to maximize the use of features in the training set. The loss function is defined as Sigmoid-BCELoss. We use the RMSProp optimizer to update the network parameters. The initial learning rate is set to 10 −5 and the decay rate is set to 10 −8 . Use a batch size of 5. Set the epoch of training times to 40, and select the network with the least loss as the best network.
The third step is to use imaris software for 3D reconstruction after the segmentation results are located according to the axis.

Comparison Methods
We compare the proposed method with (1) the method without extracting the region near the axis; (2) The manual labeling methods of different people. For quantitative evaluation, the comparison was made from four dimensions: MIoU, MRecall, MPrecision and manual participation time. The definitions of MIoU, MRecall and MPrecision are as follows: Where a is the number of categories except the background. In this paper, a=1; TP is a true positive, FP is a false positive FN is a false negative. Manual participation time is defined as the manual time required to obtain the 3D reconstruction results of the myelinated axon. Table 1 shows the evaluation of the results using the three methods. The performance of the proposed method in MIoU, MRecall and MPrecision is higher than that of the conventional U-Net segmentation method, and is similar to that of manual marking but the time cost of human participation in our method is much lower than that of pure manual marking method.  Figure 5 shows the results of labeling time as a function of the number of myelinated axons labeled when using our proposed method compared to manual labeling. The horizontal axis represents the number of myelinated axons labeled (unit: number), and the vertical axis represents the total time of manual participation of myelinated axons labeled with this number (unit: hours). As can be seen from Figure 5, when the number of axons is greater than 1, the proposed method will take much less time than pure manual labeling, and with the increase of the number of axons, the proposed method will take less time.