C++ – design of an accelerated "gradient descent" algorithm.

C++ – design of an accelerated "gradient descent" algorithm.

HomeCarlo WoodC++ – design of an accelerated "gradient descent" algorithm.
C++ – design of an accelerated "gradient descent" algorithm.
ChannelPublish DateThumbnail & View CountDownload Video
Channel AvatarPublish Date not found Thumbnail
0 Views
I think we need a /"rewrite/". Today I'm going to start again /"from scratch/".
The Main Ideas(tm) are still valid: we will use cubic fits to guess if there is a local extreme between samples, but the project is currently collapsing under its own complexity and needs a new and better way to organize all the data. implied.

My main New Insight(tm) for today is that I should store all samples in a deque that is wrapped in an "ExtremeChain" class representing a series of adjacent local extremes. It's not even really necessary to have a story. This approach will make each sample instance unique, without ever invalidating pointers. Each `ExtremeChain` also represents an interval: the leftmost and rightmost samples stored in the object. If two of these "ExtremeChains" overlap, but do not contain equivalent extremes (which should never happen), then they should only overlap on one side and not include the extreme of the other chain. In this case, the outer ends of this side must be of opposite type and adjacent. In other words, we can merge the two “ExtremeChain” objects into one in this case, and we should do so. Therefore, it can be assumed that two distinct "ExtremeChain" objects generally do not have overlapping intervals. Therefore, they can be ordered and put into a linked list. This linked list is stored and maintained by the Algorithm object.

DESCRIPTION OF THE TESTED FUNCTION:

On the left side, the function under test behaves like a parabola with a minimum. Going to the right, it never reaches this minimum, because at some point we abruptly (but smoothly, using a rapidly changing sigmoid) switch from the parabola to a sine wave. The amplitude and phase of the sine wave were manually adjusted so that the transition did not make "steps". However, there is a sudden, quite sharp curvature in the graph. It is this problem which constitutes the global minimum. The sine wave is exponentially attenuated to the right, so each minimum, to the right, is shallower. See this image: https://gyazo.com/aee0f28c55c5e4b180bd4afdb7538b21

You can follow the progress of the project as a whole here: https://github.com/CarloWood/machine-learning
discord: https://discord.gg/kbR83pPNpv

Please take the opportunity to connect and share this video with your friends and family if you find it useful.