Introduction

Biological sensory systems, which can transduce stimuli (external or internal) into neural signals, are the interface between the human and the physical world to interpret the information and create our perception of the world1. As a part of the sensory system, the somatosensory system is special and vital in the human body2,3. It is not only associated with the sense of touch, but also includes parallel receptors and nerve pathways for the sensations of temperature, body position and movement, and pain. This complex system can provide sensory feedback to the central nervous system and respond to changes in the body. Benefiting from the somatosensory functions, the possibility of an injury on our bodies can be avoided or minimized4. Taking somatosensory feedback of the hand as an example, when a strong stimulus exceeding the threshold level is applied to the hand, the nociceptors in the somatosensory system will be activated to feel pain and trigger the body’s feedback to avoid further injury5. In this process, the sensation and feedback on the stimuli form a closed-loop, which is highly efficient and significant for humans and living creatures to interact with the environment6,7.

Bionic perception systems integrated with different sensors could perceive different external stimuli information, which have great potential in communicating with complex environments, recognizing objects, and engaging in social interaction. Traditional sensory systems relying on microprocessors and external circuits show high power consumption faced with the explosive growth of data8. Recently, the advancement in flexible electronics and neuromorphic electronics has opened up opportunities to construct artificial perception systems to emulate biological functions. Until now, neuromorphic devices with various structures, materials, and mechanisms have been proposed. such as memristors9,10, synaptic electrolyte gated transistors11,12, ferroelectric transistor13, and phase change memory14. There have been several reviews on the research progress of flexible artificial synapses in recent years11,15,16. Also, parts of the biological perception functions have been emulated by using neuromorphic devices. For example, Kim et al. proposed a bioinspired afferent nerve constructed with flexible pressure sensors, ring oscillators, and a synaptic transistor, which can detect, convert, and integrate pressure information17. Lee et al. reported an organic optoelectronic sensorimotor synaptic device that can realize the light-responsive actuation of artificial muscle actuators18. Furthermore, He et al. demonstrated an artificial somatic reflex arc that can mimic the all-or-none law in the human nervous system19. However, artificial systems that can mimic the somatosensory feedback functions have not been demonstrated yet.

Herein, we developed an artificial somatosensory system with both tactile perception and instant feedback capabilities by mimicking the human somatosensory feedback pathways. In order to achieve the somatosensory feedback functions, the flexible tactile sensors, synaptic transistor, artificial muscle, and the coupling circuit are used in our system. The spatio-temporal information-processing functions are realized by the combination of multiple sensors and multi-gate synaptic transistors. The instant feedback function was realized using an artificial muscle and a comparator in the system. The research outcome provides a special paradigm for the next generation of bionic tactile perception systems.

Results and discussion

Bioinspired artificial somatosensory system

Tactile perception and instant feedback of the somatosensory system of the human are complex activities involving the perception system, nervous system, and effector. They are mainly composed of mechanoreceptors, afferent nerves, CNS, efferent nerves, and muscle20. When the tactile stimulus is detected, the action potential will be generated and transmitted between the neurons via synapses to reach the CNS (spinal cord and brain). When weak stimuli are detected, the central nervous system can recognize the external stimulus and determine its amplitude. However, if strong stimuli are detected, the CNS will send messages to the brain to make it feel painful while recognizing it as an intense amplitude. Simultaneously, an action potential will be sent to the effector to avoid further injury as shown in Fig. 1a. Inspired by feedback pathways of the somatosensory system, an artificial somatosensory system with tactile perception and instant feedback capabilities was proposed, as shown in Fig. 1b. To achieve the somatosensory feedback functions, four main components are designed and used in the system, namely flexible tactile sensor, multi-gate synaptic transistors, coupling circuits, and artificial muscles, to sense, code, process the tactile stimulus, and trigger the artificial muscle when an intensive stimuli is detected.

Fig. 1: Artificial system in comparison with the biological somatosensory system.
figure 1

a The biological somatosensory system that is stimulated by external tactile information. Pressure applied onto the mechanoreceptors in the skin can initiate action potentials. The action potentials will be transferred by afferent nerves and combined through the synapse. Afterward, the information will be processed by CNS and transferred to the brain to feel the pain, and at the same time, an action potential will be transmitted to the effector through the efferent nerve to trigger the body’s feedback to avoid further injury. b An artificial somatosensory system that mimics the biological pain perception systems composed of several main components.

Full size image

Components of the artificial system

The schematic diagram and basic performance of the main components in the artificial system are shown in Fig. 2. For the flexible tactile sensor, carbon nanotubes/polyimide (CNT/PI) with pyramidal microstructure was used as the sensitive layer and the polyethylene terephthalate (PET) with Ag electrodes was used as the substrate, as shown in Fig. 2a21. The resistance change of the flexible tactile sensor caused by the applied force will be transformed into voltage signals with different frequencies using the Voltage-frequency conversion module (AD654). To achieve a wide response range of the sensor to applied pressure, a design of CNT/PI film using micro pyramid structures with multiple different sizes was proposed as shown in Fig. 2b. Figure 2c plots the response of the flexible tactile sensor with single and micro pyramid structures with multiple different sizes in the range of 0–30 N. The initial resistance of the device with a pyramidal microstructure with multiple different sizes (size of 10 and 25 μm) is about ten times than that of the device by using a single pyramidal microstructure (10 μm), as shown in Supplementary Fig. 1. Also, the sensitivity increased from 0.088 to 0.104 kPa−1 due to the existence of the pyramidal microstructure with multiple different sizes.

Fig. 2: The main components of the artificial somatosensory system.
figure 2

a The schematic diagram and the structure of the CNT/PI based flexible sensor with pyramidal microstructure. b SEM images of the CNT/PI film in the flexible sensor, indicating that the film with multiple different sizes of micro pyramid structures with carbon nanotube networks. c Response of the resistance variation of the flexible tactile sensor under pressures from 0 to 30 N that with different micro pyramid structures (purple curve and blue curves represent the single and multiple different sizes of micro pyramid structure, respectively. The Fitted curves (dotted blue and black lines) show the sensitivity of the device. d The schematic diagram of the In2O3/SA-based artificial synaptic transistor. e Transfer characteristics of the artificial synaptic transistor at Vds of 0.5 V. f The EPSC triggered by presynaptic spikes with different voltage pulses. g Schematic configuration of the IPMC actuator. h The current response of the IPMC actuator at an applied square wave voltage (±2.5 V, 0.1 Hz). i Bending responses of the IPMC actuator at an applied square wave voltage (±2.5 V, 2 Hz).

Full size image

In2O3 and proton-conducting Sodium Alginate (SA) are used as the channel layer and dielectric gate layer of the synaptic transistor, respectively, as shown in Fig. 2d. The detailed fabricating process and the schematic diagrams for the working mechanism of the In2O3/SA-based synaptic transistor are shown in Supplementary Fig. 2 and Supplementary Fig. 3 of Supplementary Information22. Figure 2e presents the typical transfer characteristics of the In2O3/SA-based synaptic transistor. In the transfer curve, it is clear that the device exhibits a large on/off ratio of about 105 and a counterclockwise hysteresis, which are the foundations for the emulation of synaptic functions.

Moreover, benefitting from the flexible PI substrate, the transfer curve remains reliable under bending conditions which demonstrates great flexibility and reliability of the devices (Supplementary Fig. 4). The modulation of the channel current of the device under different presynaptic spikes with pulse widths ranging from 50 to 200 ms are shown in Fig. 2f indicating the excitatory postsynaptic potential (EPSC) of the devices. As a further imitation of synaptic function, the PPF and STDP functions of the devices can also be achieved (Supplementary Fig. 5 and Supplementary Fig. 6).

All these results imply that the flexible In2O3/SA-based synaptic transistor can be used to mimic the synaptic functions of the biological synapse and provides a foundation for the construction of the artificial somatosensory system.

Soft actuators, which include twisted fiber artificial muscles23,24, shape memory polymers (SMPs)25, and electrochemical actuators26 have received intensive attention in the fields of soft robots, sensors, and intelligent control27,28. As an important type of electrochemical actuator, the ionic polymer-metal composites (IPMC) actuator was employed as the artificial muscle in this system due to its advantages in low operating voltage and large deformation. The IPMC actuator was fabricated based on the SWCNTs, PVDF(HFP), and EMIMBF4, as shown in Fig. 2g, and the detailed process is shown in the Experimental section of the Supplementary Information. The actuation behavior of the artificial muscle is investigated by applying an input voltage signal (±3.0 V, 0.1 Hz) as shown in Fig. 2h. The bending mechanism of the IPMC actuator is due to the double-layer capacitance of the SWCNTs. In detail, when a voltage is applied to the actuator, cations, and anions in the gel electrolyte will be transferred to the cathode and anode layers, which leads to the formation of electrical-double layers. As a result, swelling of the cathode layer and shrinkage of the anode layer occurs. Consequently, the actuator bends to the anode side. The bending motion performance of the actuator was evaluated using an Electrochemical workstation and laser displacement sensors, as shown in Supplementary Fig. 7. A 40 mm*4 mm-wide thin strip was attached to the clip. The displacement of the actuator was tested by the laser displacement sensors under an alternating square wave (±3.0 V, 2 Hz). The largest displacement of the devices is about 0.3 mm. Besides, this process is repeated for 13 cycles as shown in Fig. 2i, which indicates the good reversibility and reliability of the IPMC actuator.

Spatio-temporal tactile perception functions

Various Mechanoreceptors, such as Meissner corpuscle, Pacinian corpuscle, Merkle complex, and Ruffini corpuscle play an essential role in the sensation of mechanical stimuli. As shown in Fig. 3a. Slowly-adapting type I and type II (SA-I, II) in the Merkel and Ruffini endings, which can produce sustained action potentials to respond to static pressure. The frequency of the action potentials rely on the the form of stimulus, and it will be transmitted by synapse to the brain, where the information is further processed29. The exist of the SA-I and II can lead the sensory organ sensitive to the intensity, time, and location of pressure. Inspired by the of SA-I and II in biological perception system, the flexible tactile sensor and the flexible synaptic transistor were integrated using the oscillator to achieve neural-like tactile perception. In this system, the voltage-frequency conversion module (AD654) and its peripheral circuit were used as the oscillator to convert analogy signals of the sensor to frequency signals, and the schematic diagram of the system is shown in Supplementary Fig. 8. Then, the recognition and tactile information processing are realized with the frequency signal as the pre-synaptic stimuli of the synaptic devices.

Fig. 3: The temporal information-processing function of the artificial system.
figure 3

a Schematic diagram of the function of slow adapting receptors. b The output voltage-pressure curve of the oscillator triggered by the flexible tactile sensor, indicating the pulse frequency can be modulated by the applied force. Line with different colors correspond to the magnitude and duration of pressure, I, II, III, and IV correspond to (0.39 N, 0.5 s), (6.87 N, 0.5 s), (24.6 N, 0.5 s), and (24.6 N, 1 s), respectively. c The postsynaptic current curves of the flexible synaptic transistor connecting with the flexible tactile sensor and oscillator under different magnitude of pressure, indicating that our system can achieve the EPSC functions triggered by different pressure. The blue, orange and purple curves represent the magnitudes of the pressure of 15.2, 25.1, and 30.4 N, respectively. d The EPSC curves of our fabricated systems under various pressure durations with the same magnitude of 25.1 N, indicating that the EPSCs increasing with the pressure durations. The purple, orange, and green curves indicate the duration of the pressure of 1, 1.9, and 2.8 s, respectively. e The ratio between the maximum value and the initial value of Ipost for different pressure magnitudes and durations.

Full size image

Figure 3b shows the output voltage signals of the oscillator triggered by the flexible tactile sensor with various stimuli. It is clear that the pulse frequency can be modulated by the applied force, indicating the conversion of pressure change to frequency change. Figure 3c shows the postsynaptic current (Ipost) of the flexible synaptic transistor stimulated by different pressures for the system, which indicates that the prepared system can achieve the EPSC functions triggered by different pressure similar to the biological tactile perception. Benefitting from the plasticity of the synaptic devices, our artificial perception systems containing synaptic devices can distinguish not only the magnitude but also the duration of pressure. Figure 3d shows the EPSCs of the systems stimulated by the stimuli with various durations at the same magnitude (25.1 N, 1.0 s, 1.9 s, and 2.8 s, respectively). As the duration increases from 1 to 2.8 s, there is a clear increase in the EPSCs.

To further explore the processing capabilities of the system for the intensity of the signals, the ratio between the maximum value and the initial value of Ipost (Am/Ai) for pressures with different magnitudes and durations are analyzed and plotted in Fig. 3e. The ratios of Am/Ai increase with the magnitudes or durations of pressures. The results indicate that the system is able to sense not only the magnitude but also the temporal information of stimuli, which will contribute to mimicking biological perception systems.

In addition to the temporal information, the determination of the spatiotemporal stimuli is also of great significance in biological sensory systems. The complex integration of multiple spatiotemporal is beneficial to the recognization of the objects, discrimination of texture, and reaction appropriately in social exchange30,31, as shown in Fig. 4a. In this work, the process was mimicked by using multiple sensors and multi-gate synaptic transistors, as shown in Fig. 4b. As described above, the drain current (Id) of the transistor is controlled by the ions from the electrolyte layer accumulated at the channel/electrolyte interface under the gate electric field. As a result, the gate-to-channel distances play a dominating role in the modulation of device conductivity. Therefore, a multi-gate synaptic transistor with different gate-to-channel distances was designed to investigate its influence on the EPSCs as shown in Fig. 4c, d. Figure 4e showed the transfer characteristics of the device with different gate-to-channel distances (Vd is 500 mV, Vg sweeps from −2.5 to 4 V, and the gate-to-channel distance of G300, G400, G500, and G600 are 300, 400, 500, and 600 μm, respectively). The drain current in the accumulation region decreases as the increase of gate-to-channel distances, which proves the control ability of the different gates. In order to further investigate the plasticity of the flexible synaptic transistor, gate voltage pulse (3 V, 100 ms) was applied in the multi-gate terminals and the EPSC response was recorded as shown in Supplementary Fig. 9. It is clear that the amplitude of the EPSC triggered by the same voltage spike decreases with the increase of the gate-to-channel distance, which proved that the devices have the potential to distinguish spatial information. To achieve the spatial tactile perceptual discrimination and integration capability of simulated biological skin, the prepared flexible sensors (S1 to S4) were connected to the multi-gate terminals of the synaptic transistors (S1 to G300, S2 to G400, S3 to G500, and S4 to G600, respectively) to simulate tactile stimuli at different locations, as shown in Fig. 4b. The stimuli were applied to the sensor of different locations and the EPSCs were tested from the synaptic transistors. From Fig. 4f, the EPSCs varies with the stimuli applied at different locations with the same duration and magnitude (32.5 N, 2 s). In order to describe the relationship between the location of the pressure and the postsynaptic response, the ratio of the maximum value of EPSC (Amax) to the initial value (Ai) is defined as L as shown in Supplementary Table 1. The gradual increase of L with the distance proved that the system contained multiple sensors and multi-gate synaptic transistors that can simulate dendritic discrimination of different spatiotemporal input sequences and is sensitive to spatial stimuli. In addition to the above spatial and temporal information resolution functions, spatial information integration is another important advantage of the tactile perception system based on artificial synaptic devices, which is different from ordinary tactile sensors. This process can be demonstrated in our systems using a weight, as shown in Fig. 4g. Weight was used to stimulate the flexible sensors and the sensors were connected to different gate terminals of the artificial synaptic device using AD654. When weights of the same mass were placed on the two sensors one by one, the resistance of the sensor changed and different EPSCs were achieved in the artificial synaptic transistor because of different gate distances. When the weights were placed on the two sensors at the same time, an increased EPSCs was obtained, which is the sum of the EPSCs obtained when weights are placed on each sensor individually. The information integration from two different gates results in a change in the EPSC amplitude, which can emulate spatial information integration correlated outputs.

Fig. 4: The spatiotemporal distinction function of the artificial system.
figure 4

a The integration of multiple spatiotemporal correlated tactile sensory stimuli of the skins. b The artificial system consists of multiple sensors and multi-gate synaptic transistors which are capable of spatiotemporal distinction. c, d are the schematic and photos of the multi-gate synaptic transistors. e The transfer characteristic curve (Id–Vg) of the synaptic transistors with different gate distances. f The EPSC response of the multi-gate terminals synaptic transistors in the artificial system when the multiple sensors are applied the same pressure. g The spatial information integration functions of the artificial system. A weight was used to stimulate the flexible sensors and the sensors were connected to different gate terminals of the artificial synaptic device using the oscillator. The information integration from two different gates results in a change in the EPSC amplitude, which can emulate spatial information integration correlated outputs.

Full size image

Instant feedback and actuation functions

Except for tactile perception and signal processing, feedback is also an important function to mimic the somatosensory system. The pain feedback function depends on the free nerve endings throughout the body and follows ‘All or none’ laws. The nerves and muscles only have two states (maximum response or none at all) in the all-or-none law, and the response cannot be graded by grading the intensity or duration of the stimulus32,33. As a proof-of-concept, an artificial somatosensory system with instant feedback and actuation ability that follows ‘All or none’ laws was designed and constructed using an IPMC. The system contained a flexible tactile sensor, an oscillator, a synaptic transistor, an I–V TIA, a comparator, and an artificial muscle (Fig. 5a), which can sense, process the external tactile stimulus, and trigger the artificial muscle when intense stimuli are sensed. Figure 5b shows the EPSC responses of the synaptic transistor triggered by voltage pulses with different pulse widths (3 V, 50 ms, 100 ms, and 150 ms, respectively). The increase of the EPSCs with the pulse width provides a basis for the determination of the threshold current and stimulates artificial muscles. The output signals of the synaptic transistors were converted by an I–V TIA and a comparator to stimulate artificial muscles. The schematic circuit diagram of the I–V TIA and comparator are shown in Supplementary Figs. 10 and 11, respectively. Figure 5c shows the output signal of the TIA (blue curve) and the comparator (red curve) when the flexible tactile in the artificial system is stimulated by different pressures. The blue curve indicates that the output signals of the artificial synaptic transistor can be accurately amplified into voltage signals. The red curve demonstrates that the artificial system can process the signals generated by different pressures using a comparator and determine whether the pressure reaches the threshold to stimulate the artificial muscles. Figure 5d shows the optical photograph of the artificial muscles in the artificial system stimulated by different pressures, which indicated that the IPMC can be triggered controllably by different tactile stimulation (the result can also be seen in Supplementary Movie 1). The above results show that our system can imitate the biological somatosensory system and realize the function of instant feedback.

Fig. 5: The instant feedback functions of the artificial system.
figure 5

a The schematic of the artificial somatosensory system composing of several main components. b EPSC response of the synaptic transistor triggered by different gate voltage spikes. c The output signal of the TIA (blue curve) and the comparator (red curve) when the artificial system is under different pressure. d The optical photograph of the artificial muscles when the artificial system is under different pressure.

Full size image

Inspired by biological somatosensory systems, an artificial somatosensory system with tactile perception and instant feedback functions was demonstrated in this work. Owing to the coupling of the multiple flexible tactile sensors and multi-gate synaptic transistors, the system can achieve the temporal and spatiotemporal information-processing ability for the tactile stimulus. Artificial muscle can be triggered controllably by different tactile stimulation. The system can mimic the somatosensory feedback functions, which shows the ability in imitating the biological somatosensory system. the research outcome shows a potential to provide a promising strategy for next-generation bionic tactile perception systems.

Methods

Preparation of flexible tactile sensor

Si mold with different micro-pyramids was prepared by conventional photolithography. Afterward, 5 mg SWCNTs were added in 100 mL N, N-dimethylformamide (DMF) and dispersed ultrasonically for 2.5 h. Then spray-coat the solution on the Si mold on a 150 °C hot plate (IKA, HP4). Afterward, the prepared polyimide (PI) solution diluting with NMP was spinning coated on Si mold to obtain the SWCNT/PI thin films with a pyramid pattern. Next, the Si mold covered with SWCNT/PI was cured at 270 °C for 1 h. PET substrate with Ag electrodes was prepared by screen printing method. A sealant was used to attach the SWCNT/PI thin films with pyramid microstructure to PET substrate with Ag electrodes and finally obtain the flexible tactile sensor. The detailed fabrication process of PI films is similar to our previous work21,34.

Preparation of the flexible synaptic transistors

A diluted polyimide (PI) solution was spin-coated on the cleaned glass and cured on the hot plate (IKA, HP4) at 270 °C for 1 h to obtain the PI film. Then electrodes (Ti/Au) were deposited on the PI substrate by a traditional photolithography method. The aqueous In2O3 precursor solution was prepared by dissolving 0.3 M indium nitrate hydrate (In(NO3) ∙ H2O) into deionized (DI) water and then stirred vigorously for 10 min. In2O3 film was obtained by spin-coating the precursor solution in PI substrate and annealing at 300 °C under ambient atmosphere for 1 h. Finally, 5 μL 2% wt SA solution was dropped on top of the active regions as a dielectric layer. The sample was cured on a hot plate at 50 °C for 30 min. The detailed fabrication process was described in our previous work22.

Preparation of the IPMC actuators

The IPMC actuators were prepared according to the methods in the literature35,36. Typically, PVDF(HFP), SWCNTs, and EMIMBF4 were added in turn to DMF and dispersed using ultrasonication to obtain the electrode solution. The mixture solution was cast on the rubber mold in glass (45 mm*45 mm*5 mm), and evaporating the solvent (60 °C, 12 h) to obtain the electrode film. The gel electrolyte film was prepared using the PVDF(HFP) and EMIMBF4 in the same method. The IPMC actuator film was fabricated by hot-pressing the electrode film and gel electrolyte film in 130 °C and 3 min.

Preparation of the circuit

AD654 was used in the construction of the pressure-to-frequency converter. The flexible tactile sensor was connected to the signal input terminal. The change of resistance can cause the change of the input voltage, and finally leads to the output of voltage signals with different frequencies (Supplementary Fig. 8).

AD825 was used as the main chip of the I–V transimpedance amplifier module, which used a T-shaped feedback network to convert a small current into a voltage signal. An adjustable inverting amplifier NE5532 was used in the second stage to amplify the voltage signal with a suitable range.

LM393 and S8050 were used in the comparator judgment circuit. The non-inverting (+IN) input is tied to a reference voltage (Vref), and the inverting (−IN) input is tied to the input signal (Vin).

Device characterization and system testing

The surface morphology of PI/SWCNT film was investigated by scanning electron microscope (SEM, Hitachi Regulus S-4800) at an accelerating voltage of 5 kV. The mechanical test of the flexible tactile sensor was tested by a customized apparatus including a source meter (Keithley 2602), a variable resistor, a pressure inductor, and a translation stage (Beijing Optical Century Instrument Co., LTD. Series SC100 stepper motor controllers). The electrical measurements of the In2O3-based synaptic transistor were performed using the Agilent B1500A semiconductor parameter analyzer. The electrical measurement in the circuit was tested using the oscilloscope (Tektronix DPO2012B). Current–voltage curves of the IPMC were obtained using an Electrochemical workstation (CHI990D). Displacement measurement of the IPMC was obtained using the laser displacement sensors (Keyence, LK-H052) and a waveform generator (Agilent 33510B).