Hello.
In case you do have low Testosterone levels, then the best treatment is Testosterone replacement itself. Clomiphene is not a good choice here.
We also need to know the cause of low Testosterone, which will need tests like LH, FSH, Prolactin and if required, a MRI scan for pituitary.
Bouts of panic and anxiety may happen due to sudden fall in Testosetrone levels, but not necessarily due to cortisol. A rise in FSH, LH and some reasons, unknown can lead to such symptoms.
So I suggest that you get your LH, FSH and Prolactin levels checked as the first step.
If after Testosterone supplementation, your anxiety does not subside, you will need to consult a psychiatrist to evaluate your panic and anxiety and for appropriate management.
Let me know if you have any other queries.
Thank you.