Architecture that can manipulate 3D or 2D representations in its latent space?

Is there an existing neural network-based architecture that can hold 3D or 2D representations in its latent space and update it in the same single pass (i.e. not compressing the representation in their final layer to be fed back to their first layer for their