Well the furthest back we can see so far is the cosmic background radiation. This was emitted when the universe went transparent, when the first atoms of hydrogen and helium formed from the hot plasma as it cooled down. That's before the first galaxies and stars.
Some think that by studying the cosmic background radiation, we might be able to see structures imprinted on it from a previous universe before this one.
We can also try to simulate earlier conditions in the big bang in particle accelerators.