In stateful state the sessions of the application will be maintained until it notifies to close the session. Each session will have particular service.
For example in Gx interface, when PCEF sends initial message (CCR-I) to PCRF a session will be formed. PCRF will reply with the same session-id CCA-I.
Now at time of CCR-U, RAR and if any event will happen in between everything would happen in the created session id.
After the termination request message (CCR-T) the session will get delete. So this all was for stateful.
For stateless there is no need to maintain any session state.
Eg. :- S6A interface is Stateless interface. In this we don't maintain any sessions. Just send Update Location Request (ULR) from MME to HSS. HSS will reply for the message with same session Update Location Answer(ULA) and clears the session on both sides.
If you want more details to clear your vision. I can explain you with more examples.