The lock core is free to rotate within the lock cylinder, but it constrained by the key wafers and retained by the last wafer. That special retainer wafer snaps open and prevents the lock core from being extracted from the cylinder. See how it sticks up when a key is inserted into the core:
Think of it as if it were a snap ring; it functions in the same manner that a snap ring would prevent a shaft from moving (because the snap ring bangs up against something). To extract the lock core, the retainer wafer is pushed inwards so that it retracts into the lock core and then the core can be removed.
In your case, that retainer wafer is not doing its job because maybe it is broken off. Or maybe it is not snapping up into position after the lock core having been removed once in its life. When you examine the lock mechanism, it will become quickly apparent how this functions, and equally apparent if the retainer is broken or missing. Compare a correctly functioning lock to a failing lock will help you to see what is going on.
