What is a Moment of Inertia?
The moment of inertia of a plane area about a given axis describes how difficult it is to change its angular motion about that axis (another way to put it is how resistant the object is to bending and torsional? stresses) . Therefore, it encompasses not just how much mass the object has overall, but how far each bit of mass is from the axis. The farther out the object's mass is, the more rotational inertia the object has, and the more force is required to change its rotation rate. This concept is made very clear in the parallel axis theorem.
In mathematical terms, the moment of inertia can be defined about any given axis as: I_x = \int y^2dA I_y = \int x^2dA 
where:
 x & y = the coordinates of the differential element dA multiplied by the square of the distance from a designated reference axis (normally the reference axis will be taken about the edge or through the centroid of the shape).
 dA = a very small element of area
Note: Moments of inertia can also be known as second moments of area.
Example of How to Calculate a Moment of Inertia
Finding I_{x}
Figure 1: Variables used for calculating Moment of Inertia
Figure 1 shows an example of a rectangle with width b and height h. The x and y axis are based off of the origin at the centroid of the rectangle (C is at \frac{h}{2} and \frac{b}{2} from the bottom right corner of the rectangle respectively). A hypothetical thin strip of area can be taken with width b and height dy (therefore dA = b * dy). Therefore I_{x} can be solved by integrating the following:
I_x = \int y^2dA = \int_{ \frac{h}{2}}^{\frac{h}{2}} y^2 b dy = \frac{bh^3}{12}
Note: I_{x} has been integrated with respect to the xaxis. The moment of inertia is dependent on what you pick as your axis. See how I_{BB} has been calculated below.
Finding I_{y}
Similarly we can choose a strip of area around the yaxis by taking dA = h * dx and get the moment of inertia with regard to the yaxis:
I_y = \int x^2dA = \int_{ \frac{b}{2}}^{\frac{b}{2}} x^2 h dx = \frac{hb^3}{12}
Finding I_{BB}
Although the most common axis' are through the centroid of the area, the moment of inertia can be taken around any arbitrary axis. For example I_{BB} can be solved by taking:
I_{BB} = \int y^2dA = \int_{0}^{h} y^2 b dy = \frac{bh^3}{3}
Common Moment of Inertia Shapes
As you can see from the examples above, it is not always easy to calculate the moment of inertia of shapes. For that reason tables have been created to speed up the process for common shapes.
The Moment of Inertia for various Structural Shapes can be found here?.
Short Excerpt on the Parallel Axis Theorem
See a full analysis of the Parallel Axis Theorem here.
Once the centroid of the shape is found, the parallel axis theorem can be used around any axis by taking: I_{A} =\sum ( I_{x} + Ad^2)

where:
 I_{A} = The moment of inertia taken about the AA axis (in^{4})
 I_{x} = The moment of inertia taken through the centroid, the xx axis (in^{4})
 A = The area of the rigid body (in^{2})
 d = the perpendicular distance between the AA axis and the xx axis (in)
Note: Looking closely at the Parallel Axis Theorem you can see that the moment of inertia of a shape will increase rapidly the further the Centroid of the area is from the axis being checked.
Resources
 H.E. Murdock, "Strength of Materials (1st Edition)", 1911
 R.C. Hibbeler, "Mechanics of Materials (7th Edition)", 2007