idwwwoqq808
Newbie
I'm using Innovus 18.10 and there is some error with placement and I could not find out why. The server CPU has 112 cores. Innovus used 48 threads, which was set by "set_multi_cpu_usage -local_cpu 48". The stack size was maximum, set by bash command "ulimit -s unlimited". I once tried running the same tcl script and same design with 16 threads and Innovus completed everything normally.
I google searched the tracking "libc.so.6(clone+0x43)" and "libpthread.so.0(+0x8609)". It looks like to be some problem with memory management.
Is there a way to solve this? What else should I do to let Innovus make use of 48 or even more local threads?
Code:
file(par.tcl) 276: set_db assign_pins_edit_in_batch false
@file(par.tcl) 277: place_opt_design
No user sequential activity specified, applying default sequential activity of "0.2" for Dynamic Power reporting.
'set_default_switching_activity' finished successfully.
*** Starting GigaPlace ***
**INFO: user set placement options
root: { place_detail_check_cut_spacing {true} place_global_activity_power_driven {false} place_global_activity_power_driven_effort {standard} place_global_clock_power_driven {true} place_global_clock_power_driven_effort {standard} place_global_cong_effort {high} place_global_place_io_pins {true} place_global_uniform_density {true}}
**INFO: user set opt options
root: { opt_all_end_points {true} opt_area_recovery {true} opt_consider_routing_congestion {auto} opt_fix_drv {false} opt_fix_fanout_load {true} opt_fix_hold_allow_overlap {auto} opt_honor_density_screen {true} opt_leakage_to_dynamic_ratio {0.2} opt_post_route_area_reclaim {none} opt_post_route_drv_recovery {auto} opt_post_route_setup_recovery {true} opt_power_effort {low}}
#optDebug: fT-E <X 2 3 1 0>
**Info: (IMPSP-307): Design contains fractional 20 cells.
*** Start delete_buffer_trees ***
Multithreaded Timing Analysis is initialized with 48 threads
Info: Detect buffers to remove automatically.
Analyzing netlist ...
Updating netlist
AAE DB initialization (MEM=3170 CPU=0:00:00.2 REAL=0:00:00.0)
*summary: 14621 instances (buffers/inverters) removed
*** Finish delete_buffer_trees (0:00:29.3) ***
**INFO: No dynamic/leakage power view specified, setting up the setup view "PVT_0P63V_100C.setup_view" as power view
** WARN: (VOLTUS_POWR-3212): The 'set_power_analysis_mode -leakage_power_view |-dynamic_power_view|-analysis_view' will be obsolete in 18.20 release. Use 'set_analysis_view -leakage <> | -dynamic <>' to set leakage and dynamic power views.
PVT_0P63V_100C.setup_view PVT_0P77V_0C.hold_view
Power Net Detected:
Voltage Name
0.00V VSS
0.63V VDD
#################################################################################
# Design Stage: PreRoute
# Design Name: ChipTop
# Design Mode: 7nm
# Analysis Mode: MMMC OCV
# Parasitics Mode: No SPEF/RCDB
# Signoff Settings: SI Off
#################################################################################
PVT_0P63V_100C.setup_view PVT_0P77V_0C.hold_view
0.00V VSS
0.63V VDD
clock_clock(500MHz) CK: assigning clock clock_clock to net axi4_mem_0_clock
Starting Levelizing
2023-Jan-10 22:56:18 (2023-Jan-11 04:56:18 GMT)
2023-Jan-10 22:56:19 (2023-Jan-11 04:56:19 GMT): 10%
2023-Jan-10 22:56:19 (2023-Jan-11 04:56:19 GMT): 20%
2023-Jan-10 22:56:19 (2023-Jan-11 04:56:19 GMT): 30%
2023-Jan-10 22:56:20 (2023-Jan-11 04:56:20 GMT): 40%
2023-Jan-10 22:56:20 (2023-Jan-11 04:56:20 GMT): 50%
2023-Jan-10 22:56:21 (2023-Jan-11 04:56:21 GMT): 60%
2023-Jan-10 22:56:21 (2023-Jan-11 04:56:21 GMT): 70%
2023-Jan-10 22:56:22 (2023-Jan-11 04:56:22 GMT): 80%
2023-Jan-10 22:56:22 (2023-Jan-11 04:56:22 GMT): 90%
Finished Levelizing
2023-Jan-10 22:56:23 (2023-Jan-11 04:56:23 GMT)
Starting Activity Propagation
2023-Jan-10 22:56:23 (2023-Jan-11 04:56:23 GMT)
** INFO: (VOLTUS_POWR-1356): No default input activity has been set. Defaulting to 0.2.
Use 'set_default_switching_activity -input_activity' command to change the default activity value.
2023-Jan-10 22:56:31 (2023-Jan-11 04:56:31 GMT): 10%
2023-Jan-10 22:56:34 (2023-Jan-11 04:56:34 GMT): 20%
Finished Activity Propagation
2023-Jan-10 22:56:43 (2023-Jan-11 04:56:43 GMT)
**Info: max transition density of cached activity is: 1e+09
Deleted 0 physical inst (cell - / prefix -).
Did not delete 207651 physical insts as they were marked preplaced.
INFO: #ExclusiveGroups=0
INFO: There are no Exclusive Groups.
Extracting standard cell pins and blockage ......
Pin and blockage extraction finished
Extracting macro/IO cell pins and blockage ......
Pin and blockage extraction finished
No user-set net weight.
Net fanout histogram:
2 : 379071 (63.9%) nets
3 : 89194 (15.0%) nets
4 - 14 : 110401 (18.6%) nets
15 - 39 : 9614 (1.6%) nets
40 - 79 : 3624 (0.6%) nets
80 - 159 : 724 (0.1%) nets
160 - 319 : 115 (0.0%) nets
320 - 639 : 49 (0.0%) nets
640 - 1279 : 1 (0.0%) nets
1280 - 2559 : 0 (0.0%) nets
2560 - 5119 : 0 (0.0%) nets
5120+ : 2 (0.0%) nets
**WARN: (IMPSP-196): User sets both -place_global_uniform_density and -place_global_initial_padding_level options. Overriding -place_global_initial_padding_level to 5.
Options: timingDriven clkGateAware ignoreScan pinGuide congEffort=high gpeffort=medium
Scan chains were not defined.
#std cell=855381 (207651 fixed + 647730 movable) #buf cell=0 #inv cell=64869 #block=535 (0 floating + 535 preplaced)
#ioInst=0 #net=592795 #term=2233984 #term/net=3.77, #fixedIo=0, #floatIo=0, #fixedPin=362, #floatPin=0
stdCell: 855381 single + 0 double + 0 multi
Total standard cell length = 1237.7966 (mm), area = 1.3368 (mm^2)
**Info: (IMPSP-307): Design contains fractional 20 cells.
Average module density = 0.136.
Density for the design = 0.136.
= stdcell_area 5315238 sites (1239939 um^2) / alloc_area 39223032 sites (9149949 um^2).
Pin Density = 0.03756.
= total # of pins 2233984 / total area 59472204.
Identified 64502 spare or floating instances, with no clusters.
Enabling multi-CPU acceleration with 16 CPU(s) for placement
=== lastAutoLevel = 12
Clock gating cells determined by native netlist tracing.
Iteration 1: Total net bbox = 1.760e+07 (1.01e+07 7.51e+06)
Est. stn bbox = 1.959e+07 (1.12e+07 8.39e+06)
cpu = 0:03:37 real = 0:03:37 mem = 4867.4M
Iteration 2: Total net bbox = 1.760e+07 (1.01e+07 7.51e+06)
Est. stn bbox = 1.959e+07 (1.12e+07 8.39e+06)
cpu = 0:00:00.8 real = 0:00:01.0 mem = 4885.2M
*** Finished SKP initialization (cpu=0:03:09, real=0:01:37)***
Iteration 3: Total net bbox = 1.254e+07 (7.39e+06 5.16e+06)
Est. stn bbox = 1.516e+07 (8.79e+06 6.37e+06)
cpu = 0:53:05 real = 0:07:00 mem = 8665.0M
Iteration 4: Total net bbox = 1.946e+07 (1.18e+07 7.63e+06)
Est. stn bbox = 2.445e+07 (1.48e+07 9.62e+06)
cpu = 1:38:56 real = 0:10:09 mem = 10053.8M
Iteration 5: Total net bbox = 1.946e+07 (1.18e+07 7.63e+06)
Est. stn bbox = 2.445e+07 (1.48e+07 9.62e+06)
cpu = 0:00:00.1 real = 0:00:00.0 mem = 10053.8M
Iteration 6: Total net bbox = 2.297e+07 (1.24e+07 1.05e+07)
Est. stn bbox = 3.073e+07 (1.62e+07 1.46e+07)
cpu = 0:49:39 real = 0:04:57 mem = 8770.5M
Iteration 7: Total net bbox = 2.321e+07 (1.27e+07 1.05e+07)
Est. stn bbox = 3.100e+07 (1.64e+07 1.46e+07)
cpu = 0:00:00.8 real = 0:00:01.0 mem = 8537.6M
Iteration 8: Total net bbox = 2.321e+07 (1.27e+07 1.05e+07)
Est. stn bbox = 3.100e+07 (1.64e+07 1.46e+07)
cpu = 0:00:01.1 real = 0:00:01.0 mem = 8537.6M
/usr/cadence/INNOVUS/INNOVUS1817/tools.lnx86/innovus/bin/64bit/innovus[0x10cee9bb]
/usr/cadence/INNOVUS/INNOVUS1817/tools.lnx86/innovus/bin/64bit/innovus(syStackTrace+0x5a)[0x10ceedb1]
/usr/cadence/INNOVUS/INNOVUS1817/tools.lnx86/innovus/bin/64bit/innovus[0x456ae95]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x14e0a2865420]
/usr/cadence/INNOVUS/INNOVUS1817/tools.lnx86/innovus/bin/64bit/innovus(_ZN8spTiming15DelayCalculator16CellArcDelayIter23calcDelayForLinearModelERKNS0_20CellArcDataPerCornerEd+0x9)[0xa0e4149]
/usr/cadence/INNOVUS/INNOVUS1817/tools.lnx86/innovus/bin/64bit/innovus(_ZN8spTiming15DelayCalculator16CellArcDelayIter4nextEv+0x608)[0xa0f56d8]
/usr/cadence/INNOVUS/INNOVUS1817/tools.lnx86/innovus/bin/64bit/innovus(_ZN8spTiming18DelayUpdateServiceIdE14processCellArcEj+0x4e)[0x9fbc7ee]
/usr/cadence/INNOVUS/INNOVUS1817/tools.lnx86/innovus/bin/64bit/innovus[0x9fdb97b]
/usr/cadence/INNOVUS/INNOVUS1817/tools.lnx86/innovus/bin/64bit/innovus(_ZN8spThread12spThreadPool6WorkerIvE4loopEv+0xc7)[0x99aea97]
/usr/cadence/INNOVUS/INNOVUS1817/tools.lnx86/lib/64bit/libnffr.so(execute_native_thread_routine+0x20)[0x14e0a8016b60]
/usr/cadence/INNOVUS/INNOVUS1817/tools.lnx86/innovus/bin/64bit/innovus[0x4814af9]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8609)[0x14e0a2859609]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x14e0a212e133]
========================================
gdb
========================================
Using: gdb
*** Stack trace:I google searched the tracking "libc.so.6(clone+0x43)" and "libpthread.so.0(+0x8609)". It looks like to be some problem with memory management.
Is there a way to solve this? What else should I do to let Innovus make use of 48 or even more local threads?
