“It will suffice therefore, either to resolved this last equation, or at least to attribute to θ a sufficiently small value, in order to obtain a new value of u inferior to u. If the new value of u is not a minimum, one will be able to deduce, by operating always in the same manner, a third still smaller value; and, by continuing thus, one will find successively some values of u more and more small, which will converge toward a minimum value of u. If the function u, which is supposed not at all to admit negative values, offers some null values, there will be able always to be determined by the preceding method, provided that one chooses conveniently the values of x, y, z, …” Augustin Cauchy, CRAS, 25, 536–538, 1847 [translation of Richard J. Pulskamp, 2010]
Surprisingly, I was not aware till a few days ago that Augustin Cauchy had made the first proposal of the gradient method to solve a minimisation problem. Which he published in a fairly vague(witness the above quote) paper in the Comptes Rendus de l’Académie des Sciences. Cauchy lived most of his (adult) life in my town of Sceaux, with his house still standing near the town centre, and now part of the nearby Marie Curie high school (named after another famous resident!).


