ALTNT's Hexo Blog

854efd16088ec0a2949395f17c9b60f9618ed3796518aea5e690afa16559fc1ffc6cfdba70664e6330a83ff79263a7d3ff15b456fed02b51dbed48301f5bee2c6dc785e7f3cf09661f7ae12954f1d51dd1766891edfd7d0639157a881313e47f1d47a411ca2d5a0f0baee9dcbfa9f998e49404d8febf3791c9c812d327b7bafe19d56a9da59b49c9ea09fd017df75f317b7bb388469373bd3383d96bb2076a49b06d7d314fd19e6c1e331da20bf36493c0a5b63cfe691754aef5251cd4b115aed4c966690eec10ed22f350cce873660c2f4c5de687217f03424becc9262371e1c5c419519670234c709f84f358aea2a0fd128c21da981acdf ...

高赞回答 1 这也是个困扰了我多年的问题： loss = a * loss1 + b * loss2 + c * loss3 怎么设置 a，b，c？ 我的经验是 loss 的尺度一般不太影响性能，除非本来主 loss 是 loss1，但是因为 b，c 设置太大了导致其他 loss 变成了主 loss。 实践上有几个调整方法： 手动把所有 loss 放缩到差不多的尺度，设 a = 1，b 和 c 取 10^k，k 选完不管了； 如果有两项 loss，可以 loss = a * loss1 + (1 - a) * loss2，通过控制一个超参数 a 调整 loss； 我试过的玄学躺平做法 loss = loss1 / loss1.detach() + loss2 / loss2.detach() + loss3 loss3.detach()，分母可能需要加 eps，一般效果不差（相当于在每一个 iteration 选定超参数 a, b, c，使得多个 loss 尺度完全一致）；进一步 loss = loss1 + loss2 / (loss2 / loss1).detach() + los ...

854efd16088ec0a2949395f17c9b60f9618ed3796518aea5e690afa16559fc1f7d749f2095f6ff6813e355b26c0a46e11157021ac3ae5f1765c0a10ef78109b65d8f3a9294d1e1acba0b85ddc7ec936dd2c93c9723144d82769f6afa0044cf34ca8560bac323d7234b817bf46af2dccc257c35197ef6e0cd5594cd7d85a43921ddd787dfaf803c717707019ad9056fa9f83aa01e792956e978335238a9d43937ace056d5a2cce53e382748be27eb0021f8c7d260eddead2c3768006cc387b55bea4e2322f665f38ee1fac083267f5a67134d7e12ecaa8a552fcfad4959f2032d6dfc752e38801054625161699e1ecfcf33aedd8831ba09f76 ...

854efd16088ec0a2949395f17c9b60f9386aedb0ff5a819cd0390912c2b2702d1787bd50453ef737f16ff88a1e3dee8a1b71cde0bfe2738aee017b35261db4c0dd20299897fab879562f13088bb18ac073716b08fa9e5bc13bc92776c26991baecc0b473d8ad562535f28f9c91c07b31ccce02a2a846104ac3e37ec535862f3f03a085a2fde5c4f582eeae9f1d3668ff9d79b1c356d7768d4de5373381e5f89f4d98a2dae8de650b54e0b4525afac9122d0a5622edcc4d098bf436e9215d7da5d5cce0babb1f96ae4068e76c00d3c57cf6bdb21407556dc67974219c670cc8f4cd086363cafbef8a37753afdb15846efa8761b0471c5320fb ...

854efd16088ec0a2949395f17c9b60f9386aedb0ff5a819cd0390912c2b2702d09373159aa72b4773c2901ef8bf3af8e5747b7afefbab7f2b076f6ad5ef72799fec69c8fa63f7d00a3b518d150fc3121036a38bf2aa36fe32d7e642fa4a0c65c1b68c65bf61bd1f8dffd6639fbe102276a26f3f0383daa8a5c5d1d7fdb6c296cd635edccfe1f9e12ab11a9299acc86de03917879f5f3f900b19ad06202dcca34fe3ca5122bc6630e0a1a806a68b8e3756078406c0f9b6e21741cb3764b67c505f21c20eac868626926f78c82ae45feff25f7e69b3e453eab776c65a91d996c8b413a79dcb9948613e9030ae0c9af26212607fb13f1f390899 ...

854efd16088ec0a2949395f17c9b60f9386aedb0ff5a819cd0390912c2b2702d1bb71c4aaccfa20a9e0b73b29b6fb413f1367d882f99222ba1727748611dd99d9a2e1dc1dbf9eb3a4d6d8b8efd69ab919bfc0031e6028a257f4a855042d93f530328c7cb190c5b4100b22979979e77e7c34a9622fae131ec470a436401b35e91878453c5874b126ffd3f145dc97a44f8484ccc2afdf6b220a584f976c89369fd0c43b730739edff5829668d7477c7d4c9957d1b407041ac071d392fc6e5a6af93cc37d413f6cd20280a15a46a3046659e57bc25dcf2c583b6ee00e4b1cd7c9209fd67be7466b2d3d0443e44dd24435067f584d605676be967 ...

854efd16088ec0a2949395f17c9b60f9618ed3796518aea5e690afa16559fc1fb797771ae20335269909bc194a028629773a1aa6a80ee502a26208086cbc7b780378e734b84c1fd5f7e4f7567d98df2aa085da52a36fbb09e19009139e2f5b3e5b7ca953d5d93796bf5c38f407c06ffe2a913e9d85335c40a25147231d4bbf655d115a8b09a69417debe59adfe6808772c1fdbe65fbf2c907ab5a736b3de86e4c5249191b673ae42517efd6bd7249535b2e1f435c206c648d67d710d11df3aa320e47fa91094c90e706423996a6efca2684014ad601739472ec88e83a35f840a6eed8d5e4c1672f74c236275e8b99ae790fd43dfc946492c3 ...

854efd16088ec0a2949395f17c9b60f9386aedb0ff5a819cd0390912c2b2702d234184895a5c49bc061bee349bb7a76e55f7528b7a89fe42f5e7518fcbfbbdfa719bdd639540af071b445dc8516a2daee2277f617a38b961d3aa06dada950a08647d3a9fc4c686323e8254ce543da16788e4e7081e2550e8817eac04cc7993d1db454eae6de657313c8e00bc02d8b454c6269f4e56c1cd20abb8de4191a616bfeaa924badafde1bdcefcb688e0dda353047d44489247b993df01546ff172ecc414e70fb8b01a9bf79910048df715a01c84d8396965a3707c0356c07bb24d4d775a4598215e025b7a1658b62e4d241862563436543e00e09a6 ...

854efd16088ec0a2949395f17c9b60f9618ed3796518aea5e690afa16559fc1fb027c7748ce414674525ebeb5e31567ba29cdfe7bd8bcb0db5dd6cbbb1515b02ccc4d3afb559ffbc1c471d9370636f7f6e3bfcaa56cadb99a312776b56a6a9f3a0bb45ef90182b71e386d0e74d253f159c5534a2b61c2e428882eb006165ca44ba0b908b5d757c8d8bd1e38323fb4eb21c9d096d8a48b9a51f25e5ac42708c8157751c1e43c1b36f1a52256e3ea8becfa476d81da983a42ee5929b85b2c7c0af0686f4b94afc3fb5ad5a3fc412e206ec6e2cc7db03d6d12bb70f99ffbdde14efdc8349454bcb42fbbebb4e708463cd86e9691f3ae6388897a ...

854efd16088ec0a2949395f17c9b60f9618ed3796518aea5e690afa16559fc1f3d073d201e9ca96ac6b613dac978477e837daea19699eef9ae69ad79db9db449f4d7607b78647b67da5531cd30e2e8f509d58a0c2e3a08c3006d6f7f74f868933683d8715d88e03177f1e0eac4485e63e62b3b4c87960bf19ce5d5ed9f29c4dcfe60c2c113cb4ef6e79eee16aa9e03a614feac8d5cb37e6365280c21b0016cc8eca4c9bccee777142105d8e6d77c3ec4eea6045971edaf319adb75cbe9f5bfe705e7979e7df1c55e3e47df0243b793a2729e661108db29662dcef81766b76302855fa0ef19c1fa1fb2d4c4e1c44f55153a742d38ba13b597f ...