The Moon's orbit (its circular path around the Earth) is indeed getting larger, at a rate of about 3.8 centimeters per year. (The Moon's orbit has a radius of 384,000 km.) I wouldn't say that the Moon is getting closer to the Sun, specifically, though--it is getting farther from the Earth, so, when it's in the part of its orbit closest to the Sun, it's closer, but when it's in the part of its orbit farthest from the Sun, it's farther away.
The reason for the increase is that the Moon raises tides on the Earth. Because the side of the Earth that faces the Moon is closer, it feels a stronger pull of gravity than the center of the Earth. Similarly, the part of the Earth facing away from the Moon feels less gravity than the center of the Earth. This effect stretches the Earth a bit, making it a little bit oblong. We call the parts that stick out "tidal bulges." The actual solid body of the Earth is distorted a few centimeters, but the most noticable effect is the tides raised on the ocean.
Now, all mass exerts a gravitational force, and the tidal bulges on the Earth exert a gravitational pull on the Moon. Because the Earth rotates faster (once every 24 hours) than the Moon orbits (once every 27.3 days) the bulge tries to "speed up" the Moon, and pull it ahead in its orbit. The Moon is also pulling back on the tidal bulge of the Earth, slowing the Earth's rotation. Tidal friction, caused by the movement of the tidal bulge around the Earth, takes energy out of the Earth and puts it into the Moon's orbit, making the Moon's orbit bigger (but, a bit pardoxically, the Moon actually moves slower!).
The Earth's rotation is slowing down because of this. One hundred years from now, the day will be 2 milliseconds longer than it is now.
This same process took place billions of years ago--but the Moon was slowed down by the tides raised on it by the Earth. That's why the Moon always keeps the same face pointed toward the Earth. Because the Earth is so much larger than the Moon, this process, called tidal locking, took place very quickly, in a few tens of millions of years.
Many physicists considered the effects of tides on the Earth-Moon system. However, George Howard Darwin (Charles Darwin's son) was the first person to work out, in a mathematical way, how the Moon's orbit would evolve due to tidal friction, in the late 19th century. He is usually credited with the invention of the modern theory of tidal evolution.
So that's where the idea came from, but how was it first measured? The answer is quite complicated, but I've tried to give the best answer I can, based on a little research into the history of the question.
There are three ways for us to actually measure the effects of tidal friction.
* Measure the change in the length of the lunar month over time.
This can be accomplished by examining the thickness of tidal deposits preserved in rocks, called tidal rhythmites, which can be billions of years old, although measurements only exist for rhythmites that are 900 million years old. As far as I can find (I am not a geologist!) these measurements have only been done since the early 90's.
* Measure the change in the distance between the Earth and the Moon.
This is accomplished in modern times by bouncing lasers off reflectors left on the surface of the Moon by the Apollo astronauts. Less accurate measurements were obtained in the early 70's.
* Measure the change in the rotational period of the Earth over time.
Nowadays, the rotation of the Earth is measured using the Very Long Baseline Interferometry, a technique using many radio telescopes a great distance apart. With VLBI, the positions of quasars (tiny, distant, radio-bright objects) can be measured very accuarately. Since the rotating Earth carries the antennas along, these measurements can tell us the rotation speed of the Earth very accurately.
However, the change in the Earth's rotational period was first measured using eclipses, of all things. Astronomers who studied the timing of eclipses over many centuries found that the Moon seemed to be accelerating in its orbit, but what was actually happening was the the Earth's rotation was slowing down. The effect was first noticed by Edmund Halley in 1695, and first measured by Richard Dunthorne in 1748--though neither one really understood what they were seeing.