Abstract for Conference- already accepted
In recent times, there has been a considerable rise in numerical computer simulations in nearly every area of science and engineering. Many of these applications are very resource demanding and require powerful compute hardware to perform. As CPU clock speeds stall out around 3 GHz, many processor manufacturers have switched to multicore systems to increase performance. But even these styles have their limitations (power consumption, heat dissipation). By making use of GPU programming languages (OpenCL, CUDA), one can harness the power of hardware already contained in a desktop system, and obtain over an order of magnitude gains in performance, all things considered. In my work I will apply these aforementioned programming techniques to illustrate their power in a Gravitational Wave Source Modelling Application.
New semester
After an eventful winter where Dr. Khanna and I submitted our second paper for publication, we’re now on the road to finishing the OpenCL project. Before doing this, we are going to tie off the remaining loose ends from the CUDA project. At this time, we have the Lax-Wendroff wave evolution code and the Teukolsky equation/Source term code as separate entities and these need to be combined into one working file (no easy task, very demanding from a memory standpoint). Once this is complete we’ll be able to benchmark the entire wave evolution code against the CPU, and have some concrete data with which to compare our OpenCL results (still to come) against. For the OpenCL portion of the research, the Lax-Wendroff iterative scheme that is used in the wave modeling portion of the project needs to be translated into openCL and then combined with the Teukolsky code into one fully parallelized and accelerated modeling application. Once we have this, we’ll really be able to fully compare the power of OpenCL with that of CUDA and base it against the serialized straight processor code.
EUREKA!!
After quite a struggle getting a handle on openCL, Dr. Khanna and myself finally have a working CPU and GPU version of the code! This is fantastic as it is literally one of the very first working openCL code pieces (of any use that is, since the others are merely adding, multiplication and “hello world” type programs) in the world!
We have however a strange problem. Since the parallelized portion of the code is launched and run in a kernel, programming get a bit tricky. When running the the kernel on the CPU, there are no worries since the kernel is in the same scope as the main code. When dealing with the GPU however, get ready to get your hands dirty. Sine the GPU is not in the same scope as the host-run code, no libraries can be appended to the kernel. This means difficult debugging, no printf statements to check your place and worst of all (at least in our case) no user defined data types. OpenCL has no predefined method of representing complex numbers (which are VERY important in calculating the source term of the Teukolsky equation). In C and C++ operator overloading is supported. The portion of code run on the host also supports user defined data types. But not the kernel! Nope. This left us high and dry for awhile. We then put maple to work in physically separating all of the real and imaginary components so that we can deal with them all separately. This took maple over two days to do (non stop) and left us with over 2 million terms (as opposed to roughly 4000)!!! To alleviate this, we had to set the spin of the black hole equal to zero to even get a reasonable amount (and it’s not very reasonable as it is). Setting the spin equal to zero changes the type of black hole we’re dealing with from a rotating (Kerr) black hole to a stationary (Schwarzschild) black hole, which is not what we want. This predicament is ongoing but looks to be the only challenge left in this small portion of the big project!!
progress report
Not too much new to report on. The code writing is coming along very well. At this time I have a working product (from a debugging standpoint) aside from determining the global work size of the data being used for the teukolsky source term calculation. Once this has been determined/debuged, I will begin writing the kernel that actually does the calculation and will be able to begin experimenting with the actual code and comparing it to CUDA and things of that matter.
A brief introduction
Before diving head first into the development of a quadruple precision variable library, it is essential to understand the reason why developing one is needed in the first place. In the following paragraphs, I’ll explain why such a set is needed.
Our sun is a relatively small star in comparison to the other stars that we know of. But for this explanation, our sun will prove to be just large enough.
In Einstein’s general theory of relativity, he proved that gravity is a consequence of the curvature of spacetime. Another important finding in this is that mass has the ability to alter the curvature of spacetime. Our sun, compared to the earth is quite massive. On the surface of the sun, hydrogen is being converted to helium by means of nuclear fusion. While the sun is composed almost entirely of hydrogen, the total amount of hydrogen on the sun is finite – meaning it will eventually run out.
For the most part (at least for the purpose of this explanation) the sun remains relatively the same size. Gravity more or less keeps the sun together inspite of all the energy it is releasing. So, it is safe to say that the force the sun exhibits outward through nuclear fusion is cancelled out by the force of gravity (keeping it the same size). To clarify; the sun attempts to completely explode and gravity holds it together.
But let’s jump a billion years into the future, when the sun is beginning to run out of hydrogen. Since the law of conservation of matter applies (matter can be neither created nor destroyed) the sun won’t actually lose any mass. But as the sun begins to run out of hydrogen, the amount of force the sun produces away from it self as a result of nuclear fusion begins to lessen. The amount of force inward does not change, as the sun still has the same mass. But now, since the amount of inward force is greater than the amount of outward force on the sun, gravity will actually begin to compact the sun. Very similar to pushing the leaves down in a leaf bag when the bag “looks full.”
In accordance with Einstein’s theory of relativity, the force due to gravity is greatest at the center of the massive object. As one gets further from the center, the force due to gravity lessens.
Now, back to our futuristic sun. As the sun creates less and less outward force, gravity will begin to compact the sun more and more. Since the sun’s mass is not changing, the gravity close to the sun becomes stronger and stronger. As the gravity close to the sun becomes stronger, it will begin to actually slow the propagation of light causing what is known as a red shift.
As time progresses, the force outward will continue to decrease and the force due to gravity will continue to compact the sun further and further. Eventually the sun will compact to the point where we can begin to examine the sun on an atomic level. In the nucleus of an atom, there exist protons, neutrons, and electrons. The neutron being the most massive of the three. Both protons and electrons can exist on their own, but a neutron will decay if it is left on its’ own. When a neutron decays, it decays into a proton, an electron, and several sub atomic particles. This process is known as beta decay. This process also works in reverse. If a proton and an electron are pushed together with enough force, the two will become one neutron.
As gravity compacts the sun further and further as the outward force created by the sun becomes less and less, the red shift will become greater and greater. The sun will appear to become darker and darker as the propagation of light becomes slower and slower.
Let’s jump a head a little bit. The force due to gravity has now compacted the sun to the point where it is a single point in space – a singularity. The nuclei of the atoms on the sun have been compacted to the point where all the electrons and protons on the sun have been pushed together with enough force to make the sun completely composed of neutrons. Since the sun has lost no mass, the force due to gravity near the sun is immense. The power of this force is so much, that not even light can escape. The photons – light carrying particles – are being physically re directed towards the center of the sun. This is effectively what is known as a black hole. The force due to gravity is so powerful that nothing can escape, including light.
Very little is known about black holes (since no one has ever experienced one). But it is known that there are massive black holes at the center of galaxies throughout the universe.
Orbiting planets travel in an elliptical pattern. How elliptical this orbit is is called the eccentricity of the planets orbit. This phenomena is what leads me to the beginning of my project.
When a particle orbits a black hole, this elliptical orbit causes the particle to be closer to the center of the black hole at certain points than at others. Since the force due to gravity around a black hole is so high, when the particle gets near the black hole as a result of its orbit, some of the energy of the particle is lost due to the friction between the planet and the force due to gravity. Over time, if the particle continues to lose bits and pieces of its orbital energy, the radius of the particles orbit will decrease. As the radius decreases, the particle will be under the influence of a stronger and stronger force due to gravity and lose more and more energy, eventually being completely enveloped by the black hole. The point at which the force due to gravity around the black hole is so powerful that nothing can escape is at a distance known as the Schwarzschild Radius.
The process of this enveloping of the particle is called an extreme mass ratio inspiral or EMRI due to the immense mass of the black hole in comparison to the particle that was orbiting it. When the particle becomes completely enveloped by the it has been proven that gravitational waves will be emitted (think of the law of conservation of energy- the energy of the particle doesn’t just disappear). Because of the extreme mass-ratio, the small object orbiting around the central supermassive black hole can be modeled as a small structure-less object, and the problem can be addressed within black hole perturbation theory. It is in this context that the Teukolsky equation becomes of considerable importance.
The Teukolsky equation is quoted Dr. Gaurav Khanna in our research paper titled An Exploration of CUDA and CBEA for a Gravitational Wave Source-Modelling Application:
“The Teukolsky equation is essentially a linear wave equation inKerr space-time geometry, with the small object acting as the generator of the gravitational waves. Thus, to numerically model an EMRI scenario, you must solve the inhomogeneous Teukolsky equation, in the time-domain.”
The Teukolsky equation is:
The right side of this equation is known at the source term. The source term plays an important role in explaining the characteristics of the EMRI. In the source term exists the quantity T. This is a representative symbol for a 4000 term calculation that is the heart of the solving of this equation. Solving such an equation, is nearly impossible by hand. In order to solve it, a computer must be used (unless you have ALOT of time on your hands). Such a mathematically intensive application is overly demanding even on a computer. This provides the need for hardward accelerators which shine in applications as intense as this. It is in this, where the heart of my project lies. Not only is it pertinent to get results as quickly as possible, but the accuracy of the results is of the highest priority. Double precision variables, when used in this calculation have been used to increase the accuracy of the calculations. These variables however, create a considerable performance loss in in terms of time when used on any platform (with respect to using a single precision variable). But even double precision variables are not accurate enough for this application. For computers, there exists a library for quadruple precision variables. These variables fall within the realm of “enough” accuracy for this type of application. The amount of time it would take to compute the solution to the source term and ultimately the Teukolsky equation using quadruple precision variables in a standard computing environment is astronomical. The need for hardware accelerators almost becomes a necessity when dealing with variables of this type. While the library exists for standard computers, the hardware accelerators I use, namely the GPU and CBEA processors, do not have a library for them. It is here, that I will begin: by designing this library to make use of quadruple precision variables. When the library has been successfully compiled, I will use this library with the hardware accelerators and compare the performance gains with these accelerators as opposed to using a standard computing environment.
Hello world!
Welcome to WordPress.com. This is your first post. Edit or delete it and start blogging!
