User talk:AliShug

Welcome!

A cup of warm tea to welcome you!

Hello, AliShug, and welcome to Wikipedia! Thank you for your contributions. I hope you are enjoying editing and want to continue. Some useful pages to visit are:

• The five pillars of Wikipedia
• Contributing to Wikipedia
• Tutorial
• How to edit a page and How to develop articles
• How to create your first article (using the Article Wizard if you wish)
• Simplified Manual of Style
• Wikipedia Teahouse (a user-friendly help forum)
• The Wikipedia Adventure (a fun game-like tour to help get you oriented within Wikipedia)

You can sign your messages on talk pages using four tildes (~~~~); this will automatically insert your username and the date.

If you need any help, check out Wikipedia:Questions, ask me on my talk page, or ask your question on this page and then place {{Help me}} before the question. We're so glad you're here! 7&6=thirteen (☎) 7&6=thirteen (☎) 22:01, 29 January 2016 (UTC)

Manifold and residual neural net

You have requested some clarification on manifolds and residual neural networks.

Assume you have one or more very complex shapes in hyperspace, possibly entangled, and you try to implement this in a simplified neural net for some purpose. (Often we will implement an alternate function, like an inverse, but this is close enough.) The neural net does not have the necessary capacity to fully implement the hypershape(s), so when the net tries to regularize the highly complex hypershape(s) the shape will go through a nonlinear dimensionality reduction. When the residual neural net expand, then its capacity increases, and then a more complex hypershape can be implemented. Because some knowledge exists from the previous learning, the network will stick closer to the (noisy) hypershape(s) given by the training data.

When using residual neural nets, we may run into a situation where the initial simplified neural net produce a simplified implementation of the manifold that can't be solved. That can happen when a manifold completely contain another manifold, unless I'm mistaken.

The term manifold is used for the hypershape, but I'm not sure where (and if) it is formally defined. The article Manifold regularization links to Manifold, but I'm not sure "manifold" in ML has all the properties from the mathematical definition. (Seems like they do…)

The blog post Colah's blog: Neural Networks, Manifolds, and Topology has some interesting insights. Jeblad (talk) 21:40, 6 March 2019 (UTC)