In the Sama Veda, in the portion relating to Udgita Vidya (one of the 32 Vidyas mentioned in the Upanishads for meditation on Brahman), Lakshmi Hayagriva is clearly indicated.
There are many passages in the Rig Veda praising Him, like the Vagabrini Suktam. In the context of Brahma Yajna, His Greatness is declared in the Yajur Veda.
The Santikam and Paushtikam, two great divisions of the Atharva Veda, adumbrate the glory of Hayagriva.
The portion called Apratiratha in the Rig Veda, relating to overcoming obstacles, has been held to refer wholly to Lord Hayagriva by Rishis in Kalpa Sutras.
There is an independent Upanishad named after Hayagriva and relating to Him.
That seers like Angiras and Atharva meditated on Hayagriva and obtained various fruits from Him is news contained in the Veda itself. Sage Dadhichi was likewise His devotee.
When Yajnavalkya prayed to the Lord that he should intuit portions of the Veda not known to others, the Lord appeared before him in the form of a horse (as Hayagriva) and fulfilled his desire.
Cited from Saptagiri (August, 1994),
"Hayagriva Stotra of Vedanta Deshika" article by L. Shrinivasan