Verilog/VHDL Verification Assistant

[MrCiffa]

Hi everyone!

I'm pretty new in the electronics space and I have a background in ai language models. Last year I fall in love with the chip industry and started digging into the verification process of hardware engineers.

Now I'm building a platform where engineers can connect their codebase via a GitHub App and select modules for whom generate automatically testbenches with AI and run verilator simulations on the cloud in a web collaborative environment where multiple engineers can check the logs.

What are your thoughts about my project? Any sincere response is welcome, thank you!

 

[dpaul]

Just like designs, testbenches are also proprietary stuff, so I would never use the above platform for my work.

Yes for students or hobbyists it may be interesting.

But then again, whether it is SV or VHDL, writing GOOD test-benches is also an art in iteslf. I PERSONALLY do not like the idea of AI generating testbenches. It eats up an engineer's knowledge to a great extent. I rould rather go with what has been continuting in this domain for years, the design engineer should be writing his own testbench for unit testing, and later, verification engineers should be exercising the entire design.

 

[MrCiffa]

Hey dpaul thanks for answering! So you don't see a future where AI assist engineers writing code? I think having an assistant that builds boilerplates and pieces of code for you could be helpful to achieve more.

 

[andre_luis]

I think having an assistant that builds boilerplates and pieces of code for you could be helpful to achieve more

ChatGPT already do this so well...or, at least, most of times.

 

[MrCiffa]

Yeah but I was thinking about a more holistic solution, where while generating the testbench you can directly run simulations, check waveforms with colleagues etc etc

 

[SynthWorks]

For VHDL, if you combined it with a verification methodology, such as OSVVM (current VHDL market leader) and ran it on GHDL, that would be interesting. OSVVM is at https://github.com/OSVVM/OsvvmLibraries.

You are not too clear about your capability. Can you create tests that exercise all states of statemachines? Can you create tests that use OSVVM's model independent transaction API that talks to the verification components and is some what readable? Can you create verification components that use OSVVM's model independent transactions and OSVVM's error reporting mechanisms?

If you can or are interested in working toward the above, you can reach me at jim at synthworks dot com.

 

[MrCiffa]

I see what you are saying, that is exactly what I would like to do. Combine AI copilot code capabilities with verification of in real time. I will write to you

 

[ThisIsNotSam]

Honestly, LLMs struggle to generate some rather simple design constructs. Maybe if you are doing software, you can tolerate some mistakes here and there and use it as copilot aid. But ASICs? That is a whole different ballgame.

No serious company would adopt an AI-assisted verification methodology given the current limitations. Verification has to be a trusted process. You cannot have false positives or false negatives. It would be a burden on designers to figure out what is real and what is not real.

My two cents.

 

[MrCiffa]

I see your concerns about adding AI to a critical step such us the functional verification. I think that LLMs can be useful for templates and in that case sometimes ChatGPT is still not good enough. My idea now after multiple feedbacks leans more towards a VSCode plugin like GitHub Copilot but specialised only to be helpful on HDL: generating templates, providing IPs and debugging models. Doing so the HW engineer is still in the cockpit but its job can be speed up by a factor of 2-3x.

Do you guys have some thoughts on what would be helpful for you from a HW copilot?

 

[MrCiffa]

I saw primis dot ai product that is working towards this direction