The castle was built between 1100 and 1120 by Mangold von Sulmetingen who later changed his name to include the element von Neuffen.[1] The first documentary evidence dates from 1198. At that time the castle was still in possession of the family von Neuffen, a member of which was the Minnesänger Gottfried von Neifen.[2] The castle went into the possession of the Lords of Weinsberg at the end of the 13th century who sold it on to the Counts of Württemberg in 1301. The castle proved its defensive worth in 1312 when, during the internal strife within the Holy Roman Empire following the election of Henry VII as Holy Roman Emperor, it could not be conquered.