Sir James Caldwell, 4th Baronet

Sir James Caldwell, 4th Baronet, Count of Milan (c.1720 – February 1784) was an Anglo-Irish soldier and author.

Caldwell was the son of Sir John Caldwell, 3rd Baronet and Anne Trench. He graduated from Trinity College Dublin in 1737 and was made a freeman of Derry in 1741. In the early 1740s he undertook the Grand Tour, during which he enlisted as an officer in the Austrian Imperial Army. He served as aide-de-camp to Charles Emmanuel III of Sardinia, and was employed chiefly in negotiating with the Kingdom of Great Britain. He was made Count of Milan by Empress Maria Theresa in 1749, having inherited his father's baronetcy in 1744. Unwilling to take the oath of allegiance to Austria, Caldwell declined the post of chamberlain to Maria Theresa and returned to Ireland in 1750. In 1753 Caldwell was elected to be a Fellow of the Royal Society.

He was appointed colonel of a regiment of militia foot and High Sheriff of Fermanagh in 1756. Between 1759 and 1763 he raised and maintained a regiment of light horse at his own expense, which was used in the suppression of Whiteboys activity. In 1762 he was made a member of the Privy Council of Ireland. He improved the estate at Castle Caldwell and gave land for the construction of a Roman Catholic church.

Caldwell was a recorder of parliamentary speeches, publishing accounts of the activity in the House of Commons of Great Britain in 1762 and of the Irish House of Commons from 1763 and 1764.

He married Elizabeth Hort, daughter of Josiah Hort. He was succeeded in his title by his eldest son, John Caldwell (1756–1830).