Law & Legal: Should I be concerned and should I take any further action?

You can either contact HR and deny it was ever said or retain an attorney and find out exactly what your rights are.