An international organization of researchers is available to step in with specialist expertise.
By Rob Meyer

If you are redesigning an airplane wing and need to know more about the turbulence flow around it, Naveed Durrani and his colleagues at the University of Sheffield’s mechanical engineering department in England will tell you that the experimental approach, while still the most trusted method for aerodynamics research, is prohibitively expensive compared to their stock in trade, computational fluid dynamics.

Durrani’s group, headed by Ning Qin, a professor in the Department of Mechanical Engineering, is actively involved in flow control, optimization, and numerical simulation of aerodynamic flows. Durrani looks to lab experiments to serve as a reality check on the computational models he is developing for his doctoral thesis on dynamic grid detached-eddy simulations. By Durrani’s estimate, his experiments in CFD will completely test a prototype in just a matter of hours far faster than by physical testing. The flexibility of changing geometrical shape and operating conditions of the CFD simulation makes it unrivaled. That’s assuming, though, that there is adequate computational power to fully validate computational models before they are put to work on simulations, and therein lies the story.

Cutting-edge CFD research like that being done by Ning’s group is tapping into unprecedented computing resources. In this case, it’s the computational power offered by HECToR (High-End Computing Tera-
scale Resources, www.hector.ac.uk), the U.K.’s national supercomputing service that went live in late 2007 to provide scientific researchers like Durrani with the ability to solve problems involving terabytes of data. Durrani’s CFD simulations run on more than 1,000 processors.

Switching to this level of processing power was not simple. According to Durrani, “Before we participated in HECToR, we didn’t have the computational power to run very big cases, and our applications were built on parallel code using a small-scale 20 node cluster, bluegrid.

“We had never gone beyond this scale of 20 nodes and hadn’t grappled with the scalability of performance and portability to a larger cluster,” Durrani said. “To utilize HECToR, we had to newly learn how to compile, port, debug, and optimize code for best performance in parallel computing. And that required outside help.”

Looking ever deeper into turbulence, researchers at the
University of Sheffield have run a detached eddy simulation
of a circular cylinder at Reynolds number 1.4e+5. They
consulted Numerical Algorithms Group to fine-tune parallel
computing software.

The HECToR team turned to a research organization of computational scientists at the Numerical Algorithms Group, which sent in a Computational Science and Engineering team. The CSE team spotted and fixed two minor code errors and resolved that problem. “They guided us through the next steps of testing and switching compilers, enabling optimization options by the compiler and removing redundant code,” Durrani said.

According to Durrani, his team “saw a 70 percent performance improvement and learned what was required for scalability to make use of HECToR-sized computational power.”

The Numerical Algorithms Group, or NAG, began in 1970 as a collaborative project among four leading British universities, and officially became a company in 1976. Its principal business is making, selling, and supporting numerical software for mathematical and statistical computation. The parent organization in the U.K. is a not-for-profit, limited-by-guarantee company, and it has operating units in the United States and Japan that are conventional commercial entities.

NAG provides numerical libraries and consulting services. Its libraries include code for both distributed and shared memory environments that engineers can use with the multicore high-power computing resources. NAG’s consulting activities develop algorithms that will be added to the libraries.

CFD research using the HECToR supercomputer in Britain included a study of this transonic cavity flow at Mach number 0.85. Some simulations by Naveed Durrani have required the computational power of more than 1,000 processors.

Among the problems the HECToR team had encountered was one that NAG’s experts had seen before. Today’s multicore processors will actually make code run existing software slower if it is not reworked for better scalability.

According to Andrew Jones, vice president of NAG’s worldwide consulting service for high-power computing, “Even though basic parallel programming is becoming more common, especially in scientific communities, the skills required to get performance—e.g., scalability—are still highly specialized. Looking years ahead, when the basic ability to write multithreaded code is more widespread and when such code is commoditized, there will still be a need for highly sophisticated software engineering skills for high thread/core count programming.”

The NAG research organization has developed a range of libraries—Fortran, C, SMP, Parallel, Data Mining Components, etc.—many of which are available in MATLAB, Maple, and other interfaces commonly used in mechanical engineering. NAG has developed a total of more than 1,500 algorithms.

NAG advises anyone seeking long-life code to similarly spare no effort to ensure that routines work on each succeeding generation of hardware. NAG researchers expect that applications will be ported numerous times to new chip architectures and operating system versions.

The NAG-recommended framework for creating robust code that can be more readily ported to different architectures involves five key points:

+ ALGORITHM SELECTION. Often, there are several methods available for solving the same mathematical or statistical problems you face, but one may have advantages over another. For example, one method might incorporate its own error estimator. This feature would help the user in understanding results and would be very useful in assuring accuracy as the routine is ported to another platform. In addition, since speed and accuracy are often in competition with one another, it’s important to choose an approach that will maintain sufficient accuracy over a broad range of data as the code is ported.

+ LANGUAGE FEATURES. In coding, NAG maintains strict standards on coding techniques and shies away from non-standard language extensions. While the use of certain language features or coding techniques may save some code or improve performance, the advantage is frequently negated when the code needs to be ported. Researchers put design emphasis on portability and ensure that every stage includes rigorous peer reviews.

+ STRINGENT TEST CODE. A stringent test code exercises all areas of a subject code and determines conditions under which it will become unstable or produce inaccurate results. The results from this stringent testing help to ensure that a code behaves as documented and does not lose an unacceptable amount of accuracy in porting.

+ EXAMPLE TEST PROGRAMS. Example programs exercise the code once installed on the user’s system with data and results from a verified installation to ensure that the code is operating as designed and as a complement to the stringent test code.

+ DOCUMENTATION. Finally, there is documentation—lots of documentation, including for both installation and routine usage. It cannot be emphasized enough that documentation, as much as functionality, helps ensure a long life for software.

The complex mechanical engineering problems where breakthrough research is being done today often involve terabyte-size datasets in order to find solutions. The inherent complexities of parallelizing and optimizing code for the high-performance computing environment require a huge investment of time not available to most in the mechanical engineering field. For the foreseeable future, HPC experts will need to be relied on by mechanical engineers to get their code to run faster and without error in multicore systems. 

The same advisors that helped Durrani regear his CFD applications for better scalability have also assisted other CFD researchers affiliated with HECToR to run a mesh containing half a billion grid points that was run for over 60,000 time-steps.

Durrani and his fellow researchers at Sheffield have not perfected their simulations yet, but they are now proceeding toward that goal quickly and accurately.

Rob Meyer is CEO of the Numerical Algorithms Group (www.nag.com). Questions can be forwarded to Rmeyer@nag.com.