java - hashmap values get changed by doing operations like retainAll()

A Map (or any collection) can only contain references to objects, not the objects themselves. When you put a copy of a reference into a collection or get a reference from a collection, you are copying just the reference, not the underlying object.

When you get a set from a Map, and you modify the set, there is only one copy of that set.

NOTE: The Map is not altered when you alter a Set in it.

If you want a copy of a collection, you have to do this explicitly.