To know what is thrashing, you must first be aware of swapping and page fault. So lets start with those concepts:
First we need to know what actually happens inside your OS that makes your system behave as if it has a very large memory. In operating systems that implement a virtual memory space, the programs allocate memory from an address space that may be much larger than the actual amount of RAM the system possesses. OS decides which program's memory is actually in RAM at any specific instant of time.
Page Fault and Swapping: A page fault occurs when the memory access requested (from the virtual address space) does not map to something that is in RAM. A page must then be sent from RAM to swap, so that the requested new page can be brought from swap to RAM. This results in 2 disk I/Os. Now you might know that disk I/Os are very slow as compared to memory access.
Thrashing: Now if it happens that your system has to swap pages at such a higher rate that major chunk of CPU time is spent in swapping then this state is known as thrashing. So effectively during thrashing, the CPU spends less time in some actual productive work and more time in swapping.
Edit: A friend of mine recently pointed out that the answer is incomplete so I am updating it now.
Now the effects of thrashing and also the extent to which thrashing occurs will be decided by the type of page replacement policy.
1. Global Page Replacement: The paging algorithm is applied to all the pages of the memory regardless of which process "owns" them. A page fault in one process may cause a replacement from any process in memory. Thus, the size of a partition may vary randomly.
2. Local Page Replacement: The memory is divided into partitions of a predetermined size for each process and the paging algorithm is applied independently for each region. A process can only use pages in its partition.
What happens after Thrashing starts?
If global page replacement is used, situations worsens very quickly. CPU thinks that CPU utilization is decreasing, so it tries to increase the degree of multiprogramming. Hence bringing more processes inside memory, which in effect increases the thrashing and brings down CPU utilization further down. The CPU notices that utilization is going further down, so it increases the degree of multiprogramming further and the cycle continues.
The solution can be local page replacement where a process can only be allocated pages in its own region in memory. If the swaps of a process increase also, the overall CPU utilization does not decrease much. If other transactions have enough page frames in the partitions they occupy, they will continue to be processed efficiently.
