Synapses are important biological structures that serve to transmit information between neurons and are thought to be the sites of learning and memory. Yet, it has remained enigmatic how memory can be retained for years while synaptic components turnover over the course of hours. Similarly, when learning a complex task, the received feedback seems most often global and poorly informative. It is then quite unclear how it can serve to properly adjust the strengths of numerous different synapses, the so-called "credit-assignment problem''. After recalling some elements of synapse biophysics and some relevant experimental findings, I will discuss our recent work in collaboration with the teams of A Triller and B Barbour at IBENS, aiming at addressing these two puzzles.