Wsi helps bring neural computers a step closer



   Neural networks have been intensely researched during the past 
few  years  and  they  are set to provide  a  new  direction  for 
computing in the 1990s.
   At  a recent technology exhibition in London,  engineers  from 
Hitachi's  central  research laboratories showed  that  by  using 
wafer  scale integration (wsi) for the implementation  they  have 
produced  a  system  they believe is of  unprecedented  size  and 
speed.
   The  system  is  based on the wsi of  digital  neural  network 
circuits with up to 144 neurons on each wafer.  The wafers are of 
5in  diameter  with the neurons and  their  interconnections,  or 
synapses,  being created using gate array methodology  and  0.8um 
technology.  Eight wafers provide up to 1152 neurons.
   Neural  networks are modelled on the form of the brain.   Each 
neuron  produces an output as a threshold function of the sum  of 
its weighted inputs.  That output is then transmitted to the next 
'layer' of neurons, where it is in turn weighted and summed  with 
other neural outputs.
   The  pattern  of connection between the input  layer  and  the 
output  layer  and  the  weights  assigned  to  each   connection 
determine the result produced from a given input.  The ability to 
alter  weights  in response to feedback allows a  network  to  be 
trained  to  generate  an  appropriate  answer  to  a  particular 
problem.
   Neural  networks  have shown promise in  solving  problems  of 
optimisation  and recognition.  But the inherent  parallelism  of 
neural  networks is not taken advantage of when run  as  software 
simulations  on conventional serial processing  computers.   This 
results in much slower operations.
   In  hardware, researchers are pursuing analogue  and  digital, 
and  optical  or electronic networks.  The most  general  network 
architecture  is  the Hopfield network, in which  each  neuron  -
neurons  can  be disconnected by setting the synaptic  weight  to 
zero.   But  where  the synapse  connections  are  permanent  and 
discrete  their  number rises with the square of  the  number  of 
neurons.
   Each  neuron  must  be able to store  a  synaptic  weight  for 
connections  from  every neuron but is obviously  unwieldy.   The 
solution to this chosen by Hitachi is a time multiplexed  digital 
bus.
   Hitachi's  engineers describe the system as  a  neurocomputer, 
and  envision it running as an accelerator or coprocessor  for  a 
conventional computer or workstation host.  The company hopes  to 
bring such a system to market within two years.
   The  engineers  responsible for the  neurocomputer  feel  that 
their combination of digital neural networks and wsi has, for the 
first time, produced a network of useful complexity and speed.
   They  say that moves to cell based layout and more  aggressive 
design rules could get them up to 1000 neurons per wafer quickly.  
There is also scope to improve the efficiency of gate utilisation 
from the currently required 2000 per neuron to 1000 per neuron.
   The  system is contained in a frame measuring 12 x 8.3 x  9in.  
Operating  from  a 5v supply, each wafer dissipates  5W  and  the 
system as a whole approximately 50W with forced air cooling  used 
to remove heat.
   The  neurocomputer  is digital in operation with  each  neuron 
producing 1 9bit result.  Up to 64 synapse weights of 8 bits  are 
stored  in  local sram at each neuron.  Each  wafer  contains  60 
gate arrays of 100,000 gate complexity.  Of these, 48 are used to 
implement three neurons per gate array along with the  associated 
bus  connections  and  memory.  The remaining  12  are  used  for 
buffering bus interconnections.
   To  provide a Hopfield network architecture, each neural  gate 
array  contains  its  own global bus which links to  two  of  its 
nearest  neighbours.   This  bus joins a main bus  block  at  the 
median line of the wafer, and from there it is taken off-wafer to 
a backplane which connects all the wafers.
   The  bus is a 9 bit multiplexed bus allowing any neuron to  be 
connected with any other neuron in the system.  The bus  protocol 
includes  a broadcast facility allowing a neuron to be linked  to 
all others simultaneously.
   This  time  multiplexed  bus  means  that  only  one   synapse 
weighting circuit is required per neuron.  Summation is performed 
by adding the weighted connections on successive cycles.  This is 
much  slower  than a fully and discreetly connected  network  but 
still  allows  neurons to operate in  parallel,  providing  speed 
benefits over the software simulation.
   As  for  learning  performance,  Hitachi  claims  2.3  billion 
connection  updates per second, and a prototype system  has  been 
used to perform signature verification in 2s.
   Neural networks are well suited to wsi as they are  inherently 
tolerant of individually defective neurons.  Such a neuron can be 
effectively   excised  from  the  network  by  setting  all   its 
coefficients to zero.
   Neural  networks  are  notoriously  difficult  to  control  or 
program but running one from a conventional host computer  helps.  
But don't expect a neurocomputer to cost less than $10,000 for  a 
minimal  configuration.   If  such a  configuration  has  a  1000 
artificial  neurons, what value does that put on the human  brain 
with about 10 billion neurons?