NX Nastran DMP
Distributed memory parallel processing with NX Nastran
www.siemens.com/nx
NX
Benefits
• Reduced solution time on large problems
• Reduced hardware
requirements for individual nodes on a cluster
• Decreased turnaround time for each solution; allowing you to analyze more designs under more conditions in a shorter amount of time
Summary
NX™ Nastran® DMP is an add-on module to the enterprise version of NX Nastran Basic software. The DMP license allows users to run NX Nastran with multiple processors on a cluster hardware system. Using multiple processors in parallel can significantly reduce solution run times compared to the more traditional serial solution that uses one processor. Also much higher levels of scalability can be obtained with distributed memory processing (DMP) compared to shared memory processing (SMP) which is the preferred technology on multiprocessor nodes with shared memory or with processor nodes with multiple cores.
Leveraging multiple processors
A distributed memory parallel (DMP) solution uses a cluster with multiple processors and multiple I/O channels communicating over a network. The figure below shows a typical architecture of a hardware system for running DMP. Each processor has its own memory and one or more disks.
DMP is a high level parallel processing methodology that works by partitioning the solution into domains that are distributed to different processors. The memory used for each domain is independent of the memory used by other domains. In contrast, SMP operates at a lower level and does not divide the solution into domains but rather uses parallelization for efficient math operations such as matrix factorization and multiplication. The
Network
CPU
Memory
Disk
CPU
Memory
Disk
CPU
Memory
Disk
CPU
Memory
Disk
time. Using the two methods together allows scalability to higher levels than could be obtained with either method individually. The figure shows the scalability obtained by an hierarchic partition of a car body model out to 64 processors.
Load domain partitioning is available for use in static analyses and is useful when there are many load cases (> 100) to solve.
For such solutions the forward-backward substitution (FBS), needed for each load set, can take more time to solve than the one time numerically intensive stiffness matrix decomposition. Partitioning the load domain leverages parallelization for the FBS by distributing load sets to multiple processors. Like frequency domain partitioning, load domain Frequency domain partitioning is available
for dynamic solutions. The solution is partitioned into frequency range segments that are solved separately with no commu- nication needed between the smaller solutions. For example, in an eigenvalue analysis for computing modes to 200 Hz, two domains can be used – one from 0-120 Hz, another from 120-200 Hz. The actual frequency boundaries are auto- matically computed by NX Nastran based on a modal density estimate – the goal to evenly balance the number of modes solved on each processor.
Hierarchic domain partitioning is a hybrid of geometry and frequency domain methods. It is used for modal solutions and allows for partitioning of geometry and the modal frequency domains at the same
NX Nastran DMP
processors work independently but the memory is shared. The independence of the memory usage by processors in a DMP solution allows for much greater leverag- ing of parallelization thus providing a higher level of scalability. With NX Nastran DMP, scalability levels to 64 processors have been obtained with no loss of performance.
DMP Implementations
DMP solutions involve a physical sub- division of the model solution domain into partitions that are distributed and processed separately. The user only needs to specify the number of processors, and the partitioning of the solution is done internally.
NX Nastran has the following options for partitioning of the solution domain.
• Geometric domain
• Frequency domain
• Hierarchic domain
• Load domain
• Recursive substructuring domain Geometric domain partitioning is available for static and dynamic solutions. The physical model is divided into geometry partitions that are solved on different processors. This table, in which N is the total number of physical DOFs and p is the number of processors, demonstrates the potential advantages for such
decomposition.
Not included in the table is the effect of the connections between the geometry partitions. These connections require communication through the message passing interface (MPI) between the processors. Therefore the geometry should be partitioned such that there is minimal connectivity between the partitions to minimize the communication overhead.
For this reason, NX Nastran DMP has several automated partitioning algo- rithms for determining an optimal set of geometric domains. In general, parti- tioning works better on shell dominant models where small domain boundaries can be found compared to “chunky” solid models that result in a large number of DOFs at the domain boundaries.
NX
25 20 15 10 15 0
Solution speed up
Number of CPUs
1 4 8 16 24 32 48 56 64
Not partitioned Partitioned
Problem size N n=N/p
Operation cost N3=p3n3 pn3
Remarks p3>p
1200 100.0 80.8 60.0 40.0 20.0 0.0
Speed up
Number of CPUs (Lancos)1 1
(Recurs ) 2 4 8 16 32 64 128
© 2011 Siemens Product Lifecycle Management Software Inc.
All rights reserved. Siemens and the Siemens logo are registered trademarks of Siemens AG. D-Cubed, Femap, Geolus, GO PLM, I-deas, Insight, JT, NX, Parasolid, Solid Edge, Teamcenter, Tecnomatix and Velocity Series are trademarks or registered trademarks of Siemens Product Lifecycle Management Software Inc. or its subsidiaries in the United States and in other countries. Nastran is a registered trademark of the National Aeronautics and Space Administration. All other logos, trademarks, registered trademarks or service marks used herein are the property of their respective holders.
X4 12563 10/11 C www.siemens.com/plm
Contact
Siemens Industry Software Americas +1 800 498 5351 Europe +44 (0) 1276 702000 Asia-Pacific +852 2230 3333
partitioning does not need communication between processors and is nearly linearly scalable.
Recursive substructuring is the newest DMP method. It works for modal solutions and is the most scalable solution yet. It operates on multilevel partitions of the mass and stiffness matrices by performing automated matrix reduction. As a mathe- matical reduction method, the resulting solution is approximate compared to the other DMP methods. However, experience has shown that the approximation is very good and has very little difference with an exact solution.
Good scalability of this method has been achieved out to 128 CPUs for a car body model as shown by the figure. Using the Lanczos method on a single processor as the baseline, the recursive DMP solution can solve more than 100 times faster. It is noted that the recursive method can also be run on a single processor and because substructuring leverage runs 4 times faster than Lanczos.
Solutions and domain options The table lists the NX Nastran solutions that are have enabled DMP processing and the domain decompositions are available for each.
NX
Availability
NX Nastran DMP is contained in the NX Nastran Advanced bundle or can be purchased as an add-on with any of the Nastran solution types outlined in the chart above. NX Nastran DMP is not available with the desktop versions of NX Nastran.
DMP Implementations in solution sequences
Solution Geometric Frequency Recursive Hierarchic Load
101 • •
103 • • • •
108 •
111 • • • •
112 • • • •
200 • • • •
