Advice on implementation my C++ perceptron in Verilog

[TSLexi]

Hi guys,
I'm trying to implement my C++ perceptron in Verilog

Here's the bit file

module Perceptron(x, y, z, out, error)

input x, y, z
output out

wire x, y, z
wire out
wire error
reg bias
reg weight1
reg weight2
reg weight3
reg weight4

bias = 1;
weight1 = 1;
weight2 = -1;
weight3 = -1;
weight4 = -1;

always @(x, y, z)
begin
reg sum
sum = 0
sum = (x*weight1)
sum = (y*weight2)+sum
sum = (z*weight3)+sum
out = (bias*weight4)+sum
end
if (error !=(Z or 0))
begin
reg correction
correction = .001
weight1 = (x*error*correction)+weight1
weight2 = (y*error*correction)+weight2
weight3 = (z*error*correction)+weight3
weight4 = (bias*error*correction)+weight4
end
endmodule

And here's the C++ source code: https://forums.codeguru.com/showthread.php?543187-Do-you-have-any-ideas-how-to-extend-my-Perceptron

Thanks,
Lexi

 

[TrickyDicky]

Im sure this is just fine as a simulation model. I can garantee it will not work on an FPGA though. (For a start, wheres the pipelining?)

 

[TSLexi]

Elaborate, as I'm pretty new to FPGA design. This is my first foray into hardware after developing software for a while

 

[rberek]

Well, here is a quick list of some issues you have to address.

1) You left off the semi-colons from just about every line.

2) You can't define things like reg or wire inside an always block.

3) Your sensitivity lists are incomplete. You generally have to put thee name of every signal on the right hand side of every equation in a senisitivity list. However, sensitivity lists for combinatorial blocks have not been required in Verilog for a very long time. Unless you are using ancient tools , always @(*) will suffice.

4) You are using real numbers (i.e. 0.001). These are not synthesizeable. If you really want floating point numbers, you will have to use one of the FPGA vendor's cores for that. Or you can use fixed point math if you prefer. Or write your own core.

5) You have no sequential logic coded here even though you will require it for statements like:

   sum = (y*weight2)+sum;

You are taking the old value of sum and adding the result of a multi-step operation to it, and then saving it back to itself. This requires memory, which means you have to make that operation sequential (i.e. flip-flops and a clock)

6) You mentioned that you are a software designer. The code below will work in C or C++ but will not work at all, as is, in Verilog (or VHDL).

sum = 0
sum = (x*weight1)
sum = (y*weight2)+sum
sum = (z*weight3)+sum
out = (bias*weight4)+sum

Now that you are describing hardware and not writing software, think about what the hardware will have to do. Sum will have to be stored in flip-flops, and assigning the value in one of the steps alone will take one clock. The new value of sum will be available for the next step after one clock period. So those 5 statements will take at least 5 clocks. The multiplications may each take one clock, or more. Thus it may take many clocks to get the value out after sum is ititialized. You cannot describe those operations by putting down five statements as they are. You will have to do some pipelining, as TrickyDicky alluded to. This will involve creating a finite state machine to manage the data flow through the steps of the computation. This will depend on how the multipliers will be implemented and what clock speed you are choosing to run this at. At high clock rates you will not be able to accomplish much between clocks to the number of pipeline stages increases.

It is said ad nauseum on this forum but has to mentioned again. Hardware Decription Languages look like programming languages but they are not. As the name implies, they are describing hardware. You need to know what hardware you need before you can describe it. And you need to know that doing things that you are used to in C or C++), like for loops, do not at all behave the same in an HDL.

I know, as a hardware beginner, that can be frustrating. Converting C or C++ code into hardware may not be the best place to start. You may want to put that on the backburner and try some simple tutorials to get used to combinatorial and sequential circuits, etc, then attack the code conversion when you have a bit better idea of the process.

Where I work we are forever turning C algorithms into Verilog or VHDL, but it is a slow, complicated procedure.

Best of Luck

r.b.

 

[TSLexi]

Here's my next try:

module Perceptron(x, y, z, out, error);

input x, y, z;
output out;

wire x, y, z;
wire out;
wire error;
reg correction;
reg bias;
reg weight1;
reg weight2;
reg weight3;
reg weight4;
reg weightedinput1, weightedinput2, weightedinput3, weightedbias;
bias <= 1;
weight1 <= 'd1;
weight2 <= 'd-1;
weight3 <= 'd-1;
weight4 <= 'd-1;

always @(*);
begin
weightedinput1 <= (x*weight1);
weightedinput2 <=(y*weight2);
weightedinput3 <= (z*weight3);
weightedbias <= (bias*weight4);
out  = weightedinput1+weightedinput2+weightedinput3+weightedbias;
end
if (error !=(Z or 0))
begin
correction =  'd.001;
weight1 <= (x*error*correction)+weight1;
weight2 <= (y*error*correction)+weight2;
weight3 <= (z*error*correction)+weight3;
weight4 <= (bias*error*correction)+weight4;
end
endmodule

 

[rberek]

TSLexi

Would mind answering a couple of questions?

1) Do you intend to ever implement this code in an FPGA?

2) What tools are you using to synthesize or otherwise error-check your code?

3) Could you describe for us, in words or a diagram, how your Perceptron would be represented in hardware? What is the data flow? What hardware elements (i.e. memories, flip flops, lookup tables, etc) are required to make one?

The reason I ask is that your code still does not represent hardware in any way and has many syntactical errors.

For example:

1) there are no semi-colons after always statements
2) I had mentioned that always @(*) is for combinatorial blocks. For synthesizeable RTL, combinatorial blocks may only have blocking (i.e. =) assignments
3) Sequential blocks (i.e. always @ (posedge clock)) , for synthesizeable RTL, should only use non-blocking assignments (i.e. <=)
4) Non blocking assignments should not be used outside of an always block
5) You cannot mix blocking and non-blocking assignments in an always block
6) You are still using real numbers. OK, if you just want to simulate, bad for synthesis

I suggest you check out a site like fpga4fun that has lots of code examples and tutorials that will guide you in creating proper combinatorial and sequential templates for synthesis, and give you an idea of the types of hardware that often get implemented.

I assume you will eventually want to implement this code in hardware, as your goal is to learn about hardware. In order to best do this, take a breath and take the time to get the hardware and Verilog basics down by checking out sites like fpga4fun

r.b.

 

[TSLexi]

TSLexi

Would mind answering a couple of questions?

1) Do you intend to ever implement this code in an FPGA?

2) What tools are you using to synthesize or otherwise error-check your code?

3) Could you describe for us, in words or a diagram, how your Perceptron would be represented in hardware? What is the data flow? What hardware elements (i.e. memories, flip flops, lookup tables, etc) are required to make one?

The reason I ask is that your code still does not represent hardware in any way and has many syntactical errors.

For example:

1) there are no semi-colons after always statements
2) I had mentioned that always @(*) is for combinatorial blocks. For synthesizeable RTL, combinatorial blocks may only have blocking (i.e. =) assignments
3) Sequential blocks (i.e. always @ (posedge clock)) , for synthesizeable RTL, should only use non-blocking assignments (i.e. <=)
4) Non blocking assignments should not be used outside of an always block
5) You cannot mix blocking and non-blocking assignments in an always block
6) You are still using real numbers. OK, if you just want to simulate, bad for synthesis

I suggest you check out a site like fpga4fun that has lots of code examples and tutorials that will guide you in creating proper combinatorial and sequential templates for synthesis, and give you an idea of the types of hardware that often get implemented.

I assume you will eventually want to implement this code in hardware, as your goal is to learn about hardware. In order to best do this, take a breath and take the time to get the hardware and Verilog basics down by checking out sites like fpga4fun

r.b.

Answers to your questions:

1. Yes, when I save up the money to buy the Xilinx Artix-7 FPGA AC701 Evaluation Kit.

2. Unfortunately, due to my laptop's chipset melting, and the simulator that's available for my Android phone locking it up, I don't have one currently

3. The perceptron would be implemented like this:

https://i.stack.imgur.com/KUvpQ.png

Aka, three wires representing inputs, plus a register holding a value of 1, would be multiplied by 4 weights, which are four registers initially holding a random value. Those values would then be added together, and that value is >0, the output would be 1, otherwise it'd output -1.

There would also be a register holding a learning rate value of .001. And a wire representing error, which can have values of 0, 2,-2, or Z
If the output is different from the correct output, the each weight register would add (input*error*learning rate) to itself. Same for the weight register corresponding to the bias input.

I'll check out fpga4fun. Here's a MUCH simpler bit file I was doing: https://www.edaboard.com/threads/308881/

Thanks,
Lexi

 

[TSLexi]

Okay, I learned more about Verilog, so here's what I got now:

module Perceptron(x, y, z, out, error);

input x, y, z, error;
output out;

wire x, y, z;
wire out;
wire error;

real correction;
real bias;
real weight1;
real weight2;
real weight3;
real weight4;
real weightedinput1, weightedinput2, weightedinput3, weightedbias;

initial
begin
assign weight1 = 1;
assign weight2 = -1;
assign weight3 = -1;
assign weight4 = -1;
assign bias = 1;
assign correction = .001;
end

always(x or y or z);
begin
assign weightedinput1 <= (x*weight1);
assign weightedinput2 <= (y*weight2);
assign weightedinput3 <= (z*weight3);
assign weightedbias <= (bias*weight4);

assign out  = weightedinput1+weightedinput2+weightedinput3+weightedbias;
end

always(error);
if (error !=(Z or 0))
begin
assign weight1 <= (x*error*correction)+weight1;
assign weight2 <= (y*error*correction)+weight2;
assign weight3 <= (z*error*correction)+weight3;
assign weight4 <= (bias*error*correction)+weight4;
end
endmodule

I read about the real datatype, so hopefully this will properly represent what I want this IC to do.

 

[mrflibble]

real is fine for simulation. But if you want it to work in actual hardware you'll have to do something else. May I suggest looking at how to use fixed point math in constructing that perceptron.

 

[TSLexi]

I'm going to do simulation on my CIS lab's computer until I buy a device board, and then just use an IP core to do the math.

 

[mrflibble]

Alright have fun, and don't forget to pipeliiiiiiiiine.

 

[rberek]

HI

You did learn more about Verilog but still have several syntactical errors.

1) You forgot the @ sign after the always

2) Assigns cannot occur inside always or initial statements.

3) assign statements use the blocking (=), never the non-blocking (<=) assignment. You should definitely try to understand the difference between blocking and non-blocking assignments

Now, note that this code you've written is behavioural code and is not synthesizeable at all, since it does not describe implementable hardware. When you go to implement it in an FPGA you will have to completely rewrite it.

Your diagram is a pretty high level system diagram and does not show the feedback that you verbally describe, in which the weights are modified and the result stored. This storage means you will need sequential circuitry (i.e. flip flops). The code you wrote has no sequential blocks in it.

Unfortunately Verilog chose the keyword reg to indicate an element which can store a state. And hardware folks use the term register and flip flop interchangeably. But making something a register in a Verilog file does not by itself make something synthesize into a flip flop. Later versions of Verilog have done away with the keywords wire and reg.

So you can try to simulate this to see what happens. You are doing some things I have never done, like use assign for real types, but the LRM implies it is doable, so I would be interested in your results.

When you want to implement it in your Artix 7 you will have to start from scratch and use synthesizeable sequential and combinatorial templates to implement the perceptron in hardware. Then it becomes interesting,since you will have to concern yourself with timing as well as getting correct hardware. If your clock spee dis high you may see that the multipliers with use up much of your timing budget. Then, as has been suggested by several, pipelining will become your friend.

r.b.

 

[mrflibble]

HI
Your diagram is a pretty high level system diagram and does not show the feedback that you verbally describe, in which the weights are modified and the result stored. This storage means you will need sequential circuitry (i.e. flip flops). The code you wrote has no sequential blocks in it.

Yup. The shown diagram is trivial, in fact, it's textbook. The real work is 1) designing a perceptron that supports all the required operations, and 2) all the support circuitry that controls the initialization/seeding and feedback for you. Assuming you're doing a multilayer perceptron because you want to actually do something.

 

[FvM]

Forgetting about synthesizable Verilog for a moment, it's still no usefull model.
- All input and output values must have be of the real type, too
- The weight integrator needs a timed event for reasonable operation (and there's no Z state for real variables, by the way)

I'm going to do simulation on my CIS lab's computer until I buy a device board, and then just use an IP core to do the math.

Just a bit to learn. No IP core will work with real variables. Float can be used, but not in simple behavioral statements as you are writing. Scaling the calculation to use integer or fixed point values would be the next step in a HDL design dedicated for synthesis in hardware.