{"id":928,"date":"2026-07-04T05:15:57","date_gmt":"2026-07-04T05:15:57","guid":{"rendered":"https:\/\/hyc.eshachem.com\/program\/?page_id=928"},"modified":"2026-07-04T06:13:16","modified_gmt":"2026-07-04T06:13:16","slug":"4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp","status":"publish","type":"page","link":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/","title":{"rendered":"4. \u5be6\u9a57: \u624b\u523b NLP"},"content":{"rendered":"\n<h1 class=\"wp-block-heading\" id=\"%e5%af%a6%e9%a9%97%e4%b8%80-rnn\">\u5be6\u9a57\u4e00: RNN<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">\u524d\u5f80\u4e0b\u8f09ai-lab\u7a0b\u5f0f\u78bc\uff0c\u5c07\u5176\u8cbc\u4e0agoogle colab\u9032\u884c\u5be6\u9a57!<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/github.com\/Chen11111112\/ai-lab.git\">https:\/\/github.com\/Chen11111112\/ai-lab.git<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"%e5%af%a6%e9%a9%971-1-%e9%9a%b1%e8%97%8f%e7%8b%80%e6%85%8b\">\u5be6\u9a571-1 \u2013 \u96b1\u85cf\u72c0\u614b<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u5728\u505a\u4ec0\u9ebc<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u7528\u4e00\u6bb5\u4e2d\u6587\u77ed\u6587\u8a13\u7df4\u4e00\u500b\u8ff7\u4f60 RNN\uff0c\u5b78\u7fd2\u300c\u7d66\u524d 3 \u500b\u5b57\uff0c\u731c\u7b2c 4 \u500b\u5b57\u300d\u3002<\/li>\n\n\n\n<li>\u8a13\u7df4\u5b8c\u6210\u5f8c\uff0c\u7d66\u958b\u982d\u300c\u8001\u738b\u300d\uff0c\u8b93 AI \u81ea\u52d5\u7e8c\u5beb 20 \u500b\u5b57\u3002<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u6838\u5fc3\u6982\u5ff5<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>Embedding<\/code>\uff1a\u628a\u5b57\u8f49\u6210\u5411\u91cf<\/li>\n\n\n\n<li><code>nn.RNN<\/code>\uff1a\u4f9d\u5e8f\u8655\u7406\u6642\u9593\u6b65\uff0c\u7d2f\u7a4d\u96b1\u85cf\u72c0\u614b<\/li>\n\n\n\n<li><code>BPTT<\/code>\uff08<code>loss.backward()<\/code>\uff09\uff1a\u6cbf\u6642\u9593\u53cd\u5411\u50b3\u64ad\uff0c\u66f4\u65b0\u6b0a\u91cd W<\/li>\n\n\n\n<li><code>optimizer.step()<\/code>\uff1a\u6839\u64da\u68af\u5ea6\u8abf\u6574\u53c3\u6578<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u89c0\u5bdf\u91cd\u9ede<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u7e8c\u5beb\u5167\u5bb9\u662f\u5426\u51fa\u73fe\u8a13\u7df4\u6587\u672c\u4e2d\u7684\u7247\u8a9e\uff08\u5982\u300c\u6697\u5df7\u300d\u300c\u6253\u706b\u6a5f\u300d\u300c\u5fae\u7a4d\u5206\u300d\uff09\uff1f<\/li>\n\n\n\n<li>\u6a21\u578b\u80fd\u5426\u5340\u5206\u5169\u689d\u6545\u4e8b\u7dda\uff08\u6697\u5df7 vs \u6559\u5ba4\uff09\uff1f<\/li>\n\n\n\n<li>\u5e8f\u5217\u9577\u5ea6\u50c5 3\uff0c\u6a21\u578b\u8a18\u61b6\u5f88\u77ed\uff0c\u5bb9\u6613\u300c\u5fd8\u8a18\u300d\u66f4\u65e9\u7684\u4e0a\u4e0b\u6587\u3002<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"390\" height=\"111\" src=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/3F4A3D7A-311E-498A-B781-3B3B22DBAF90.png\" alt=\"\" class=\"wp-image-889\" srcset=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/3F4A3D7A-311E-498A-B781-3B3B22DBAF90.png 390w, https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/3F4A3D7A-311E-498A-B781-3B3B22DBAF90-300x85.png 300w\" sizes=\"auto, (max-width: 390px) 100vw, 390px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u591a\u57f7\u884c\u5e7e\u6b21\u6703\u767c\u73fe\u50b3\u7d71 RNN \u7684\u96b1\u85cf\u72c0\u614b\uff08Hidden State\uff09\u7121\u6cd5\u5728\u9577\u8ddd\u96e2\u7684\u6587\u672c\u4e2d\uff0c\u7cbe\u6e96\u5206\u914d\u300e\u73fe\u5728\u8a72\u628a\u6ce8\u610f\u529b\u653e\u5728\u6697\u5df7\u9084\u662f\u6559\u5ba4\u300f\u7684\u6b0a\u91cd\uff0c\u9019\u4e5f\u6210\u4e86\u5b83\u649e\u4e0a\u6280\u8853\u9ad8\u7246\u7684\u5be6\u8b49\uff0c\u9032\u800c\u63a8\u52d5\u4e86\u5f8c\u7e8c <strong>LSTM \u9580\u63a7\u6a5f\u5236<\/strong> \u8207 <strong>Transformer \u6ce8\u610f\u529b\u6a5f\u5236<\/strong> \u7684\u8a95\u751f\u3002<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\">\n\n\n\n<h2 class=\"wp-block-heading\" id=\"%e5%af%a6%e9%a9%971-2-%e6%a2%af%e5%ba%a6%e6%b6%88%e5%a4%b1\">\u5be6\u9a571-2 \u2013 \u68af\u5ea6\u6d88\u5931<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u5728\u505a\u4ec0\u9ebc<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u8207 1-1 \u76f8\u540c\u7684 RNN \u67b6\u69cb\uff0c\u4f46\u5c07 <code>seq_length<\/code> \u5f9e <strong>3 \u62c9\u5230 30<\/strong>\u3002<\/li>\n\n\n\n<li>\u5176\u9918\u8a2d\u5b9a\u76f8\u540c\uff0c\u65b9\u4fbf\u5c0d\u7167\u3002<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u89c0\u5bdf\u91cd\u9ede<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5c0d\u6bd4 1-1 \u7684\u7e8c\u5beb\u54c1\u8cea\uff1a\u9577\u5e8f\u5217\u4e0b\uff0cRNN \u662f\u5426\u66f4\u96e3\u5b78\u5230\u6709\u6548\u6a21\u5f0f\uff1f<\/li>\n\n\n\n<li>\u751f\u6210\u7d50\u679c\u662f\u5426\u66f4\u6df7\u4e82\u3001\u91cd\u8907\uff0c\u6216\u7121\u6cd5\u5ef6\u7e8c\u5408\u7406\u8a9e\u610f\uff1f<\/li>\n\n\n\n<li><strong>\u68af\u5ea6\u6d88\u5931\uff08Vanishing Gradient\uff09<\/strong> \u7684\u76f4\u89c0\u5c55\u793a\uff1a\u5e8f\u5217\u8d8a\u9577\uff0c\u65e9\u671f\u6642\u9593\u6b65\u7684\u68af\u5ea6\u5728\u53cd\u5411\u50b3\u64ad\u4e2d\u9010\u6f38\u8870\u6e1b\uff0c\u6a21\u578b\u96e3\u4ee5\u5b78\u7fd2\u9060\u8ddd\u96e2\u4f9d\u8cf4\u3002<\/li>\n<\/ul>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\"><strong>\u5c0d\u7167\u5be6\u9a57\u5efa\u8b70<\/strong>\uff1a\u5148\u8dd1 1-1\uff0c\u518d\u8dd1 1-2\uff0c\u6bd4\u8f03\u5169\u6b21\u300cAI \u7e8c\u5beb\u300d\u7684\u5dee\u7570\u3002<\/p>\n<\/blockquote>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"390\" height=\"111\" src=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/10D09421-FCBE-4D11-986B-971062EE77E5.png\" alt=\"\" class=\"wp-image-891\" srcset=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/10D09421-FCBE-4D11-986B-971062EE77E5.png 390w, https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/10D09421-FCBE-4D11-986B-971062EE77E5-300x85.png 300w\" sizes=\"auto, (max-width: 390px) 100vw, 390px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u56e0\u70ba\u6211\u5011\u7528\u7684\u662f <code>nn.RNN<\/code>\u3002\u7576\u6642\u9593\u8ef8\u62c9\u5230 30 \u500b\u5b57\u9019\u9ebc\u9577\u6642\uff0c\u53cd\u5411\u50b3\u64ad\u7684\u8aa4\u5dee\u8a0a\u865f\u50b3\u5230\u7b2c 5 \u500b\u5b57\u5c31\u5df2\u7d93\u56e0\u70ba\u300c\u9023\u4e58\u6548\u61c9\u300d\u840e\u7e2e\u5230\u8b8a\u6210 0\uff08\u68af\u5ea6\u6d88\u5931\uff09\u4e86\u3002\u524d\u9762\u7684\u5b57\u6839\u672c\u7df4\u4e0d\u5230\u6b0a\u91cd\uff0cAI \u76f4\u63a5\u653e\u68c4\u601d\u8003\u3002<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\">\n\n\n\n<h2 class=\"wp-block-heading\" id=\"%e5%af%a6%e9%a9%971-3-lstm\">\u5be6\u9a571-3 \u2013 LSTM<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u5728\u505a\u4ec0\u9ebc<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5c07 RNN \u63db\u6210 <code>nn.LSTM<\/code>\uff0c\u5f15\u5165\u7d30\u80de\u72c0\u614b\uff08Cell State\uff09\u8207\u9598\u9580\u6a5f\u5236\u3002<\/li>\n\n\n\n<li>\u751f\u6210\u6642<strong>\u4fdd\u7559\u4e26\u50b3\u905e hidden state<\/strong>\uff08<code>state<\/code> \u8b8a\u6578\uff09\uff0c\u8b93\u6a21\u578b\u8de8\u6b65\u9a5f\u7d2f\u7a4d\u8a18\u61b6\u3002<\/li>\n\n\n\n<li>\u958b\u982d\u7dda\u7d22\u6539\u70ba\u300c\u8001\u738b\u5728\u300d\uff08\u9577\u5ea6 3\uff0c\u7b26\u5408 <code>seq_length<\/code>\uff09\u3002<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u89c0\u5bdf\u91cd\u9ede<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u7e8c\u5beb 40 \u5b57\u6642\uff0c\u662f\u5426\u80fd\u7dad\u6301\u8f03\u4e00\u81f4\u7684\u8a9e\u610f\u8d70\u5411\uff1f<\/li>\n\n\n\n<li>\u5c0d\u6bd4 1-1 \u540c\u6a23\u77ed\u5e8f\u5217\u7684 RNN\uff1aLSTM \u5728\u9700\u8981\u8de8\u591a\u6b65\u63a8\u7406\u6642\u662f\u5426\u66f4\u7a69\u5b9a\uff1f<\/li>\n\n\n\n<li>\u6ce8\u610f\u751f\u6210\u8ff4\u5708\u4e2d <code>state<\/code> \u6709\u6c92\u6709\u88ab\u6b63\u78ba\u50b3\u5165\u4e0b\u4e00\u8f2a\u2014\u2014\u9019\u662f LSTM \u8207\u666e\u901a RNN \u63a8\u7406\u6642\u7684\u95dc\u9375\u5dee\u7570\u3002<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"680\" height=\"106\" src=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/F2B6B9E6-9B48-4AE6-98D1-2EAF4BCEA4D7.png\" alt=\"\" class=\"wp-image-893\" srcset=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/F2B6B9E6-9B48-4AE6-98D1-2EAF4BCEA4D7.png 680w, https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/F2B6B9E6-9B48-4AE6-98D1-2EAF4BCEA4D7-300x47.png 300w\" sizes=\"auto, (max-width: 680px) 100vw, 680px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u9019\u5982\u679c\u662f\u539f\u672c\u7684\u666e\u901a RNN\uff0c\u5b83\u53ea\u8981\u4e00\u9032\u5165\u300c\u6559\u5ba4\u88e1\u300d\uff0c\u5c31\u518d\u4e5f\u56de\u4e0d\u4f86\u300c\u6697\u5df7\u88e1\u300d\u4e86\u3002\u4f46\u9019\u53f0\u8ff7\u4f60 LSTM \u5b83\u5beb\u5b8c\u6559\u5ba4\u7684\u5287\u60c5\u4e4b\u5f8c\uff0c<strong>Cell State\uff08\u8a18\u61b6\u9ad8\u901f\u516c\u8def\uff09\u6210\u529f\u7684\u628a\u300c\u8001\u738b\u9084\u6709\u53e6\u4e00\u500b\u6697\u5df7\u5287\u672c\u300d\u7684\u8a18\u61b6\u4e00\u8def\u50b3\u4e86\u4e0b\u4f86<\/strong>\uff0c\u6240\u4ee5\u5b83\u80fd\u5728\u5f8c\u534a\u53e5\u6210\u529f\u5207\u63db\u6642\u7a7a\uff0c\u958b\u59cb\u5beb\u300c\u8001\u738b\u5728\u6697\u5df7\u88e1\u9ede\u71c3\u4e86\u2026\u300d\u3002<\/p>\n\n\n\n<hr class=\"wp-block-separator aligncenter has-alpha-channel-opacity\">\n\n\n\n<h3 class=\"wp-block-heading\" id=\"%e4%ba%86%e4%ba%86%e4%ba%86%e4%ba%86%e4%ba%86%ef%bc%9f\">\u4e86\u4e86\u4e86\u4e86\u4e86\uff1f<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u90a3\u70ba\u4ec0\u9ebc\u5b83\u6703\u958b\u59cb\u50cf\u8df3\u91dd\u4e00\u6a23\u300c\u4e86\u4e86\u4e86\u4e86\u4e86\u4ed6\u300d\uff1f\u9019\u5728 NLP\uff08\u81ea\u7136\u8a9e\u8a00\u8655\u7406\uff09\u88e1\u53eb <strong>\u300c\u904e\u5ea6\u81ea\u4fe1\u8207\u8caa\u5a6a\u89e3\u6cd5\uff08Greedy Search\uff09\u7684\u7f3a\u9677\u300d<\/strong>\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5728\u6211\u5011\u7684\u7a0b\u5f0f\u78bc\u4e2d\uff0c\u6709\u9019\u95dc\u9375\u7684\u4e00\u884c\uff1a<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-1\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">next_char_id = torch.argmax(pred).item() <span class=\"hljs-comment\"># \u6c38\u9060\u53ea\u62ff\u6a5f\u7387\u6700\u9ad8\u7684\u90a3\u500b\u5b57<\/span>\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-1\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">\u56e0\u70ba\u6211\u5011\u7684\u53e5\u5b50\u88e1\u51fa\u73fe\u4e86\u975e\u5e38\u591a\u6b21\u7684\u300c\u4e86\u300d\uff08\u9ede\u71c3<strong>\u4e86<\/strong>\u3001\u7167\u4eae<strong>\u4e86<\/strong>\u3001\u62ff\u8d77<strong>\u4e86<\/strong>\u3001\u96e3\u5012<strong>\u4e86<\/strong>\uff09\u3002\u5c0d AI \u4f86\u8aaa\uff0c\u5728\u6a5f\u7387\u7d71\u8a08\u4e0a\uff0c<strong>\u300c\u4e86\u300d\u5f8c\u9762\u518d\u5ea6\u63a5\u300c\u4e86\u300d\u7684\u671f\u671b\u503c\u88ab\u56b4\u91cd\u653e\u5927\u4e86<\/strong>\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u9019\u4e5f\u662f\u70ba\u4ec0\u9ebc\u73fe\u4ee3\u7684 LLM \u5728\u751f\u6210\u6587\u5b57\u6642\uff0c\u6703\u52a0\u5165\u4e00\u500b\u53eb <strong>Temperature\uff08\u6eab\u5ea6\/\u96a8\u6a5f\u6027\uff09<\/strong> \u7684\u53c3\u6578\u3002<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\">\n\n\n\n<h1 class=\"wp-block-heading\" id=\"%e5%af%a6%e9%a9%97%e4%ba%8c-attention-mechanism\">\u5be6\u9a57\u4e8c \u2013 Attention Mechanism<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u5728\u505a\u4ec0\u9ebc<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u4ee5\u6975\u5c0f\u898f\u6a21\u7684\u82f1\u7ffb\u4e2d\u8cc7\u6599\uff083 \u53e5\uff09\u8a13\u7df4 Encoder-Decoder \u6a21\u578b\u3002<\/li>\n\n\n\n<li>Decoder \u7aef\u52a0\u5165 <strong>Attention<\/strong>\uff0c\u8b93\u89e3\u78bc\u6642\u80fd\u300c\u56de\u982d\u770b\u300dEncoder \u5404\u4f4d\u7f6e\u7684\u8f38\u51fa\u3002<\/li>\n\n\n\n<li>\u8a13\u7df4\u904e\u7a0b\u4e2d\uff0c\u5728\u7b2c 1\u30015\u300110\u300120\u300140\u300160\u300180\u3001100 \u8f2a\u5370\u51fa\u7ffb\u8b6f\u7d50\u679c\uff0c\u89c0\u5bdf\u6f38\u9032\u5f0f\u5b78\u7fd2\u3002<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u6838\u5fc3\u6982\u5ff5<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>Encoder (GRU)<\/code>\uff1a\u628a\u82f1\u6587\u53e5\u5b50\u58d3\u6210\u96b1\u85cf\u8868\u793a<\/li>\n\n\n\n<li><code>Decoder (GRU)<\/code>\uff1a\u9010\u5b57\u7522\u751f\u4e2d\u6587<\/li>\n\n\n\n<li><code>Attention<\/code>\uff1a<code>softmax(Q\u00b7K^T)<\/code> \u8a08\u7b97\u6b0a\u91cd\uff0c\u52a0\u6b0a\u5408\u4f75 Encoder \u8f38\u51fa\u4f5c\u70ba Context<\/li>\n\n\n\n<li><code>&lt;SOS&gt;<\/code> \/ <code>&lt;EOS&gt;<\/code>\uff1a\u6a19\u8a18\u751f\u6210\u8d77\u9ede\u8207\u7d42\u9ede<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u89c0\u5bdf\u91cd\u9ede<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>\u8a13\u7df4\u8f2a\u6b21<\/th><th>\u9810\u671f\u73fe\u8c61<\/th><\/tr><\/thead><tbody><tr><td>\u7b2c 1 \u8f2a<\/td><td>\u8f38\u51fa\u63a5\u8fd1\u4e82\u78bc\u6216\u91cd\u8907\u5b57\u5143<\/td><\/tr><tr><td>\u7b2c 5\u201320 \u8f2a<\/td><td>\u958b\u59cb\u51fa\u73fe\u6b63\u78ba\u5b57\u8a5e\u7247\u6bb5\uff08\u5982\u300c\u4f60\u300d\u300c\u540d\u300d\u300c\u5b57\u300d\uff09<\/td><\/tr><tr><td>\u7b2c 40\u2013100 \u8f2a<\/td><td>\u9010\u6f38\u63a5\u8fd1\u5b8c\u6574\u8b6f\u53e5\uff0cLoss \u6301\u7e8c\u4e0b\u964d<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u6e2c\u8a66\u53e5 <code>\"What is your name?\"<\/code> \u662f\u5426\u6700\u7d42\u7ffb\u6210\u300c\u4f60\u53eb\u4ec0\u9ebc\u540d\u5b57\uff1f\u300d\uff1f<\/li>\n\n\n\n<li>\u89c0\u5bdf Loss \u6578\u503c\u8207\u7ffb\u8b6f\u54c1\u8cea\u7684\u5c0d\u61c9\u95dc\u4fc2\u3002<\/li>\n\n\n\n<li>\u53ef\u5617\u8a66\u4fee\u6539 <code>LEARNING_RATE<\/code> \u6216 <code>EPOCHS<\/code>\uff0c\u89c0\u5bdf\u6536\u6582\u901f\u5ea6\u8b8a\u5316\u3002<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"697\" height=\"257\" src=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/79D83E94-99E0-4046-8EDE-41C3FCFC9382.png\" alt=\"\" class=\"wp-image-930\" srcset=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/79D83E94-99E0-4046-8EDE-41C3FCFC9382.png 697w, https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/79D83E94-99E0-4046-8EDE-41C3FCFC9382-300x111.png 300w\" sizes=\"auto, (max-width: 697px) 100vw, 697px\" \/><\/figure>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\">\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u3010\u7b2c 1 \u8f2a\u3011\u7ffb\u8b6f \u2500\u2500&gt; \u738b\u58de\u738b\u7d2f\u58de\u9928\u7684\u7684\u7684\u4ed6\u738b\u738b\u8089\u80b2\u9ebc\u8089\u62cd\u8089\u62cd\u8089<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u9019\u6642\u5019\u6a21\u578b\u525b\u521d\u59cb\u5316\uff0c\u6240\u6709\u7684\u77e9\u9663\u53c3\u6578\uff08Linear \u5c64\u3001Embedding \u5c64\uff09\u88e1\u5168\u662f\u96a8\u6a5f\u4e82\u6578\u3002<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li><strong>\u80e1\u4e82\u6293\u53d6\u9ad8\u983b\u5b57\uff1a<\/strong> \u6b64\u6642\u6a21\u578b\u9084\u6c92\u6709\u4efb\u4f55\u300c\u6587\u6cd5\u300d\u6216\u300c\u5c0d\u7126\u300d\u6982\u5ff5\u3002\u5b83\u53ea\u6ce8\u610f\u5230\u6578\u64da\u96c6\u88e1\u7b2c 2\u30013 \u53e5\u6709\u5927\u91cf\u95dc\u65bc\u8001\u738b\u53bb\u9ad4\u80b2\u9928\u3001\u7d2f\u58de\u4e86\u3001\u6253\u7403\u7684\u8a5e\u5f59\u3002<\/li>\n\n\n\n<li><strong>\u9577\u53e5\u58d3\u5236\u77ed\u53e5\uff1a<\/strong> \u56e0\u70ba\u7b2c 2\u30013 \u53e5\u7684\u4e2d\u6587\u9577\u5ea6\uff0815\u300111 \u500b\u5b57\uff09\u9060\u5927\u65bc\u7b2c 1 \u53e5\uff087 \u500b\u5b57\uff09\uff0c\u5728\u96a8\u6a5f\u72c0\u614b\u4e0b\uff0c\u9577\u53e5\u5b50\u7684 Token \u7522\u751f\u7684\u68af\u5ea6\u5f71\u97ff\u529b\u5728\u77e9\u9663\u88e1\u4f54\u4e86\u4e0a\u98a8\u3002\u65bc\u662f\u6a21\u578b\u5728\u89e3\u78bc\u6642\uff0c\u5927\u8166\u88e1\u5168\u662f\u7b2c 2\u30013 \u53e5\u7684\u96a8\u6a5f\u96dc\u8a0a\uff0c\u6700\u5f8c\u62fc\u6e4a\u51fa\u4e00\u4e32\u514b\u8607\u9b6f\u5f0f\u7684\u80e1\u8a00\u4e82\u8a9e\u3002<\/li>\n<\/ol>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\"><strong>\u3010\u7b2c 5 \u8f2a\u3011\u7ffb\u8b6f \u2500\u2500&gt; \u8001\uff1f\u58de\u9928\u9928\u7684\u4ed6<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u3010\u7b2c 10 \u8f2a\u3011\u7ffb\u8b6f \u2500\u2500&gt; \u8001<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u3010\u7b2c 20 \u8f2a\u3011\u7ffb\u8b6f \u2500\u2500&gt; \u9ad4\u80b2\u9928<\/strong><\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">\u9019\u662f\u6700\u7cbe\u5f69\u7684\u968e\u6bb5\uff01\u96a8\u8457 Loss \u5f9e 3.65 \u964d\u5230 2.55\uff0c\u795e\u7d93\u7db2\u8def\u958b\u59cb\u767c\u73fe\u300c\u53ea\u8981\u4e82\u8b1b\u8a71\uff0cLoss \u5c31\u6703\u88ab\u91cd\u7f70\uff08CrossEntropyLoss \u5f88\u9ad8\uff09\u300d\u3002<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li><strong>\u7b2c 5-10 \u8f2a\u7684\u300c\u64fa\u721b\uff08\u904e\u5ea6\u4fdd\u5b88\uff09\u300d\uff1a<\/strong> \u6a21\u578b\u767c\u73fe\u591a\u8aaa\u591a\u932f\uff0c\u4f46\u5b83\u9084\u6c92\u5b78\u6703\u6b63\u78ba\u7ffb\u8b6f\uff0c\u65bc\u662f\u4e7e\u8106\u300c\u5c11\u8aaa\u5c11\u932f\u300d\u3002\u5b83\u5410\u51fa\u4e00\u500b <code>\u8001<\/code> \u4e4b\u5f8c\uff0c\u5f8c\u9762\u7684\u6a5f\u7387\u76f4\u63a5\u649e\u4e0a <code>&lt;PAD&gt;<\/code>\uff080\uff09\u6216 <code>&lt;EOS&gt;<\/code>\uff0c\u9078\u64c7\u63d0\u65e9\u9589\u5634\u3002<\/li>\n\n\n\n<li><strong>\u7b2c 20 \u8f2a\u7684\u300c\u5c40\u90e8\u5c0d\u7126\u932f\u8aa4\u300d\uff1a<\/strong> \u9019\u884c <code>\u9ad4\u80b2\u9928<\/code> \u662f\u6574\u4efd\u65e5\u8a8c\u7684<strong>\u9ec3\u91d1\u6559\u6750<\/strong>\uff01\u70ba\u4ec0\u9ebc\u8f38\u5165 <code>What is your name?<\/code>\uff0cAI \u6703\u7ffb\u6210 <code>\u9ad4\u80b2\u9928<\/code>\uff1f\n<ul class=\"wp-block-list\">\n<li>\u4f86\u770b\u7b2c 2 \u53e5\u7684\u82f1\u6587\uff1a<code>\"He\", \"played\", \"badminton\", \"in\", \"the\", \"gym\"<\/code><\/li>\n\n\n\n<li>\u4f86\u770b\u7b2c 3 \u53e5\u7684\u82f1\u6587\uff1a<code>\"the\", \"gym\", \"tired\", \"him.\"<\/code><\/li>\n\n\n\n<li>\u4f60\u767c\u73fe\u4e86\u55ce\uff1f\u9019\u5169\u53e5\u90fd\u6709 <code>\"the\", \"gym\"<\/code>\u3002\u800c\u6b64\u6642\u7b2c 1 \u53e5\u662f <code>\"What\", \"is\", \"your\", \"name?\"<\/code>\u3002<\/li>\n\n\n\n<li><strong>\u6a21\u578b\u7684 Attention \u641e\u932f\u4e86\uff1a<\/strong> \u96a8\u6a5f\u6b0a\u91cd\u5728\u4fee\u6b63\u6642\uff0c\u4e0d\u5c0f\u5fc3\u628a\u7b2c 1 \u53e5\u5f8c\u6bb5 Padding \u7684 <code>0<\/code>\uff08\u7a7a\u767d\uff09\u548c\u7b2c 2\u30013 \u53e5\u7684\u7d50\u5c3e\u7279\u5fb5\u641e\u6df7\u4e86\u3002\u6b64\u6642\u7684 Attention \u6b0a\u91cd\uff08$Q \\times K$\uff09\u932f\u8aa4\u5730\u5c0d\u7126\u5230\u4e86 <code>\"gym\"<\/code> \u9019\u500b\u5b57\u4e0a\uff01\u65e2\u7136\u773c\u775b\u76ef\u8457 <code>\"gym\"<\/code>\uff0cDecoder \u7406\u6240\u7576\u7136\u5730\u5410\u51fa\u4e86 <code>\"\u9ad4\u80b2\u9928\"<\/code>\u3002<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\"><strong>\u3010\u7b2c 40 \u8f2a\u3011\u7ffb\u8b6f \u2500\u2500&gt; \u4f60\u53eb\u4ec0\u9ebc\u540d\u5b57\uff1f<\/strong><\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">\u7576 Loss \u8dcc\u7834 1.5 \u7684\u81e8\u754c\u9ede\uff08\u81e8\u754c\u9ede\u901a\u5e38\u4ee3\u8868\u6a21\u578b\u7d42\u65bc\u7406\u6e05\u4e86\u4e0d\u540c\u53e5\u5b50\u9593\u7684\u908a\u754c\uff09\uff0cAdam \u512a\u5316\u5668\u7d42\u65bc\u628a Attention \u77e9\u9663\uff08Q, K, V\uff09\u6821\u6e96\u5230\u4e86\u6b63\u78ba\u7684\u7a7a\u9593\u4f4d\u7f6e\u3002<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li><strong>\u7cbe\u6e96\u8a9e\u5e8f\u7ffb\u8f49\uff08\u5012\u88dd\u53e5\u6210\u529f\uff09\uff1a<\/strong>\u6b64\u6642\u6a21\u578b\u7684 Q\uff08\u76ee\u524d\u6211\u60f3\u5410\u51fa\u4ec0\u9ebc\u4e2d\u6587\uff09\u8207 K\uff08\u82f1\u6587\u55ae\u5b57\u7279\u5fb5\uff09\u6b63\u78ba\u78b0\u649e\u3002\u7576\u89e3\u78bc\u5668\uff08Decoder\uff09\u5e36\u6709 <code>&lt;SOS&gt;<\/code> \u555f\u52d5\u6642\uff0c\u5b83\u7684 Attention Weights \u7b97\u51fa\u4f86\uff0c\u6700\u9ad8\u5206\u4e0d\u518d\u662f\u96a8\u6a5f\u4e82\u6578\uff0c\u4e5f\u4e0d\u662f <code>\"gym\"<\/code>\uff0c\u800c\u662f\u6b7b\u6b7b\u76ef\u8457 <code>\"your\"<\/code>\u3002\u56e0\u6b64\u6210\u529f\u5410\u51fa <code>\u4f60<\/code> \u2192 <code>\u53eb<\/code>\u3002\u63a5\u8457\u8996\u7dda\u5927\u632a\u79fb\uff0c\u77ac\u9593\u79fb\u56de\u53e5\u9996\u76ef\u8457 <code>\"What\"<\/code>\uff0c\u5410\u51fa <code>\u4ec0<\/code> \u2192 <code>\u9ebc<\/code>\u3002<\/li>\n\n\n\n<li><strong>\u5b78\u6703\u81ea\u5f8b\u9589\u5634\uff08<code>&lt;EOS&gt;<\/code> \u767c\u63ee\u4f5c\u7528\uff09\uff1a<\/strong>\u66f4\u53b2\u5bb3\u7684\u662f\uff0c\u7576\u5410\u51fa <code>\uff1f<\/code> \u4e4b\u5f8c\uff0c\u56e0\u70ba\u4f60\u5728\u8a13\u7df4\u96c6\u52a0\u4e86 <code>\"&lt;EOS&gt;\"<\/code>\uff0c\u6a21\u578b\u5927\u8166\u8a08\u7b97\u51fa\u300c\u9019\u6642\u5019\u8a72\u7d50\u675f\u4e86\u300d\u7684\u6a5f\u7387\u9ad8\u9054 99%\u3002\u65bc\u662f\u5b83\u5728\u7b2c 8 \u6b65\u5410\u51fa\u4e86 <code>zh_to_int[\"&lt;EOS&gt;\"]<\/code>\u3002\u7a0b\u5f0f\u78bc\u4e2d\u7684 <code>if next_id == zh_to_int[\"&lt;EOS&gt;\"]: break<\/code> \u77ac\u9593\u88ab\u89f8\u767c\uff0c\u8ff4\u5708\u5b8c\u7f8e\u4e2d\u65b7\u3002<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">\u5f8c\u9762\u7684 60\u300180\u3001100 \u8f2a\uff0c\u96d6\u7136\u8f38\u51fa\u7684\u6587\u5b57\u4e00\u6a21\u4e00\u6a23\uff0c\u4f46 Loss \u5f9e <code>1.25<\/code> \u4e00\u8def\u66b4\u8dcc\u5230 <code>0.10<\/code>\u3002\u9019\u4ee3\u8868 AI \u5c0d\u65bc\u300c\u9019\u500b\u5b57\u5f8c\u9762\u8a72\u63a5\u54ea\u500b\u5b57\u3001\u773c\u775b\u8a72\u770b\u54ea\u88e1\u300d\u7684<strong>\u628a\u63e1\u5ea6\uff08\u6a5f\u7387\u5206\u4f48\u7684\u6ce2\u5cf0\uff09\u8b8a\u5f97\u6975\u5ea6\u81ea\u4fe1\u8207\u5c16\u92b3<\/strong>\u3002<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\">\n\n\n\n<h1 class=\"wp-block-heading\" id=\"%e5%af%a6%e9%a9%97%e4%b8%89-transformer\">\u5be6\u9a57\u4e09 \u2013 Transformer<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u5728\u505a\u4ec0\u9ebc<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5f9e\u7dad\u57fa\u767e\u79d1\u7db2\u9801\u5373\u6642\u722c\u53d6\u524d\u4e09\u6bb5\u6587\u5b57\u3002<\/li>\n\n\n\n<li>\u900f\u904e NVIDIA NIM \u96f2\u7aef API\uff0c\u547c\u53eb <strong>Llama-3.1-8B<\/strong> \u9032\u884c 2\u20133 \u53e5\u82f1\u6587\u6458\u8981\u3002<\/li>\n\n\n\n<li>\u9ad4\u9a57\u5de5\u696d\u7d1a Transformer \u7684\u5be6\u969b\u61c9\u7528\uff0c\u8207\u524d\u9762\u672c\u5730\u5c0f\u6a21\u578b\u7684\u5dee\u7570\u3002<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u4f7f\u7528\u524d\u8a2d\u5b9a<\/strong><\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u81f3 <a href=\"https:\/\/build.nvidia.com\/\">NVIDIA Build<\/a> \u7533\u8acb API Key\u3002<\/li>\n\n\n\n<li>\u958b\u555f <code>3-Transformer\/1-Transformer.py<\/code>\uff0c\u5c07\u7b2c 16 \u884c\u7684 <code>NVIDIA_API_KEY<\/code> \u66ff\u63db\u70ba\u4f60\u7684\u91d1\u9470\uff1a<\/li>\n<\/ol>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-2\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\">NVIDIA_API_KEY = <span class=\"hljs-string\">\"nvapi-\u4f60\u7684\u771f\u5be6\u91d1\u9470\"<\/span><\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-2\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<ol start=\"3\" class=\"wp-block-list\">\n<li>\u57f7\u884c\u5f8c\u53ef\u8f38\u5165\u4efb\u610f\u82f1\u6587\u7dad\u57fa\u767e\u79d1 URL\uff0c\u6216\u76f4\u63a5\u6309 Enter \u4f7f\u7528\u9810\u8a2d\u7684 AI \u689d\u76ee\u3002<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u89c0\u5bdf\u91cd\u9ede<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u722c\u87f2\u662f\u5426\u6210\u529f\u6293\u53d6\u6bb5\u843d\uff08\u6ce8\u610f\u7d42\u7aef\u6a5f\u5370\u51fa\u7684\u5b57\u5143\u6578\uff09\u3002<\/li>\n\n\n\n<li>\u6458\u8981\u662f\u5426\u6293\u4f4f\u6587\u7ae0\u6838\u5fc3\u5b9a\u7fa9\u8207\u56e0\u679c\u95dc\u4fc2\uff08\u800c\u975e\u9010\u53e5\u8907\u8ff0\uff09\uff1f<\/li>\n\n\n\n<li>\u5c0d\u6bd4\u5be6\u9a57 1\u30012 \u7684\u300c\u5f9e\u96f6\u8a13\u7df4\u300d\uff1a\u96f2\u7aef\u5927\u6a21\u578b\u5c55\u73fe\u7684\u662f<strong>\u9810\u8a13\u7df4 + \u6307\u4ee4\u5fae\u8abf<\/strong>\u5f8c\u7684\u8a9e\u610f\u7406\u89e3\u80fd\u529b\u3002<\/li>\n\n\n\n<li>\u53ef\u5617\u8a66\u4e0d\u540c\u7dad\u57fa\u9801\u9762\uff08\u5982 <code>Machine_learning<\/code>\u3001<code>Neural_network<\/code>\uff09\uff0c\u89c0\u5bdf\u6458\u8981\u54c1\u8cea\u8b8a\u5316\u3002<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"742\" height=\"378\" src=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/91C4C89C-CD45-49E5-933A-D40338259FA8.png\" alt=\"\" class=\"wp-image-937\" srcset=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/91C4C89C-CD45-49E5-933A-D40338259FA8.png 742w, https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/07\/91C4C89C-CD45-49E5-933A-D40338259FA8-300x153.png 300w\" sizes=\"auto, (max-width: 742px) 100vw, 742px\" \/><\/figure>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\">\n\n\n\n<h3 class=\"wp-block-heading\" id=\"%e8%ab%8b%e6%b1%82%e8%88%87%e6%8c%87%e4%bb%a4%e5%82%b3%e8%bc%b8%e9%9a%8e%e6%ae%b5%ef%bc%88the-instruction-prompt%ef%bc%89\">\u8acb\u6c42\u8207\u6307\u4ee4\u50b3\u8f38\u968e\u6bb5\uff08The Instruction Prompt\uff09<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u5728 3-Transformer\/1-Transformer.py \u4e2d\u6700\u8070\u660e\u7684\u4e00\u74b0\uff0c\u662f\u5728 <code>client.chat.completions.create<\/code> \u88e1\u9762\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong><code>role: system<\/code><\/strong>\uff1a\u6211\u5011\u5728\u9019\u88e1\u8ce6\u4e88\u4e86 AI \u4e00\u500b\u300c\u4eba\u683c\u300d\u3002\u6211\u5011\u544a\u8a34\u5b83\uff1a\u300c\u4f60\u662f\u4e00\u4f4d\u5091\u51fa\u7684\u6559\u6388\uff0c\u8acb\u7528 2-3 \u53e5\u8a71\u7cbe\u7149\u6587\u7ae0\uff0c\u805a\u7126\u5728\u56e0\u679c\u95dc\u4fc2\u3002\u300d\u9019\u662f\u5728\u8a2d\u5b9a Transformer \u5167\u90e8\u795e\u7d93\u5143\u6fc0\u6d3b\u7684\u65b9\u5411\uff08Bias\uff09\u3002<\/li>\n\n\n\n<li><strong><code>role: user<\/code><\/strong>\uff1a\u9019\u662f\u300c\u6ce8\u5165\u98df\u6750\u300d\u3002\u6211\u5011\u5c07\u525b\u624d\u722c\u87f2\u6293\u4e0b\u4f86\u7684\u4e00\u5927\u5768\u6587\u5b57\uff08<code>input_text<\/code>\uff09\u8ddf\u6307\u4ee4\u5408\u4f75\u5728\u4e00\u8d77\u3002<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"%e7%9f%a9%e9%99%a3%e9%81%8b%e7%ae%97%e8%88%87%e6%8e%a8%e8%ab%96%e9%9a%8e%e6%ae%b5%ef%bc%88the-transformer-reasoning%ef%bc%89\">\u77e9\u9663\u904b\u7b97\u8207\u63a8\u8ad6\u968e\u6bb5\uff08The Transformer Reasoning\uff09<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u7576\u6578\u64da\u9001\u9054 NVIDIA \u7684\u96f2\u7aef\u5f8c\uff0c\u771f\u6b63\u7684 Transformer \u6a21\u578b\u5c31\u63a5\u624b\u4e86\u3002\u9019\u767c\u751f\u4e86\u4ec0\u9ebc\u4e8b\uff1f<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>\u5411\u91cf\u5316\uff08Embedding\uff09<\/strong>\uff1a\u6a21\u578b\u5c07\u9019\u4e9b\u6587\u5b57\u8f49\u63db\u6210\u6578\u5343\u7dad\u7a7a\u9593\u4e2d\u7684\u300c\u6578\u5b57\u5411\u91cf\u300d\u3002<\/li>\n\n\n\n<li><strong>\u591a\u982d\u6ce8\u610f\u529b\u6a5f\u5236\uff08Multi-Head Attention\uff09<\/strong>\uff1a\u9019\u662f\u6838\u5fc3\uff01\u6a21\u578b\u540c\u6642\u8a08\u7b97\u9019 1000 \u591a\u500b\u5b57\u4e2d\uff0c\u6bcf\u4e00\u500b\u5b57\u8207\u5176\u4ed6\u5b57\u4e4b\u9593\u7684\u300c\u6b0a\u91cd\u300d\u3002\u5b83\u6703\u767c\u73fe\u300c\u4eba\u5de5\u667a\u6167\u300d\u3001\u300c\u5b9a\u7fa9\u300d\u3001\u300c\u767c\u5c55\u300d\u9019\u4e9b\u8a5e\u51fa\u73fe\u983b\u7387\u6700\u9ad8\uff0c\u4e14\u56e0\u679c\u95dc\u4fc2\u6700\u7dca\u5bc6\u3002<\/li>\n\n\n\n<li><strong>\u6982\u7387\u5206\u4f48\u9810\u6e2c\uff08Probability Distribution\uff09<\/strong>\uff1a\u6a21\u578b\u4e0d\u662f\u5728\u300c\u8907\u88fd\u300d\u6587\u7ae0\uff0c\u5b83\u662f\u5728\u6839\u64da\u525b\u525b\u8a08\u7b97\u51fa\u7684\u5411\u91cf\u6b0a\u91cd\uff0c<strong>\u9810\u6e2c\u4e0b\u4e00\u500b\u6700\u61c9\u8a72\u51fa\u73fe\u7684\u5b57\u662f\u4ec0\u9ebc<\/strong>\u3002\u5b83\u5728\u8166\u4e2d\u91cd\u65b0\u69cb\u601d\u4e86\u4e00\u6bb5\u65b0\u7684\u6587\u5b57\uff0c\u4f86\u56de\u61c9\u6211\u5011\u7684\u6307\u4ee4\u3002<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\">\n\n\n\n<h1 class=\"wp-block-heading\" id=\"%e7%b5%90\">\u7d50<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">\u5230\u6700\u5f8c\u4e00\u500b\u5be6\u9a57\uff0c\u5176\u5be6\u6211\u5011\u5df2\u7d93\u505a\u4e86\u4e00\u500b\u7c21\u55ae\u7248\u7684 <strong>RAG (Retrieval-Augmented Generation\uff0c\u6aa2\u7d22\u589e\u5f37\u751f\u6210)<\/strong> !<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u6574\u500b\u7bc7\u7ae0\u5230\u9019\u88e1\uff0c\u76f8\u4fe1\u4f60\u4e5f\u5df2\u7d93\u662f\u500b\u5c0f\u767dAI\u5de5\u7a0b\u5e2b\u4e86!!!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u5be6\u9a57\u4e00: RNN \u524d\u5f80\u4e0b\u8f09ai-lab\u7a0b\u5f0f\u78bc\uff0c\u5c07\u5176\u8cbc\u4e0agoogle colab\u9032\u884c\u5be6\u9a57! https:\/\/github.com\/Chen11111112\/ai-lab.git \u5be6\u9a571-1 \u2013 \u96b1\u85cf\u72c0\u614b \u5728\u505a\u4ec0\u9ebc \u6838\u5fc3\u6982\u5ff5 \u89c0\u5bdf\u91cd\u9ede \u591a\u57f7\u884c\u5e7e\u6b21\u6703\u767c\u73fe\u50b3\u7d71 RNN \u7684\u96b1\u85cf\u72c0\u614b\uff08Hidden State\uff09\u7121\u6cd5\u5728\u9577\u8ddd\u96e2\u7684\u6587\u672c\u4e2d\uff0c\u7cbe\u6e96\u5206\u914d\u300e\u73fe\u5728\u8a72\u628a\u6ce8\u610f\u529b\u653e\u5728\u6697\u5df7\u9084\u662f\u6559\u5ba4\u300f\u7684\u6b0a\u91cd\uff0c\u9019\u4e5f\u6210\u4e86\u5b83\u649e\u4e0a\u6280\u8853\u9ad8\u7246\u7684\u5be6\u8b49\uff0c\u9032\u800c\u63a8\u52d5\u4e86\u5f8c\u7e8c LSTM \u9580\u63a7\u6a5f\u5236 \u8207 Transformer \u6ce8\u610f\u529b\u6a5f\u5236 \u7684\u8a95\u751f\u3002 \u5be6\u9a571-2 \u2013 \u68af\u5ea6\u6d88\u5931 \u5728\u505a\u4ec0\u9ebc \u89c0\u5bdf\u91cd\u9ede \u5c0d\u7167\u5be6\u9a57\u5efa\u8b70\uff1a\u5148\u8dd1 1-1\uff0c\u518d\u8dd1 1-2\uff0c\u6bd4\u8f03\u5169\u6b21\u300cAI \u7e8c\u5beb\u300d\u7684\u5dee\u7570\u3002 \u56e0\u70ba\u6211\u5011\u7528\u7684\u662f nn.RNN\u3002\u7576\u6642\u9593\u8ef8\u62c9\u5230 30 \u500b\u5b57\u9019\u9ebc\u9577\u6642\uff0c\u53cd\u5411\u50b3\u64ad\u7684\u8aa4\u5dee\u8a0a\u865f\u50b3\u5230\u7b2c 5 \u500b\u5b57\u5c31\u5df2\u7d93\u56e0\u70ba\u300c\u9023\u4e58\u6548\u61c9\u300d\u840e\u7e2e\u5230\u8b8a\u6210 0\uff08\u68af\u5ea6\u6d88\u5931\uff09\u4e86\u3002\u524d\u9762\u7684\u5b57\u6839\u672c\u7df4\u4e0d\u5230\u6b0a\u91cd\uff0cAI \u76f4\u63a5\u653e\u68c4\u601d\u8003\u3002 \u5be6\u9a571-3 \u2013 LSTM \u5728\u505a\u4ec0\u9ebc \u89c0\u5bdf\u91cd\u9ede \u9019\u5982\u679c\u662f\u539f\u672c\u7684\u666e\u901a RNN\uff0c\u5b83\u53ea\u8981\u4e00\u9032\u5165\u300c\u6559\u5ba4\u88e1\u300d\uff0c\u5c31\u518d\u4e5f\u56de\u4e0d\u4f86\u300c\u6697\u5df7\u88e1\u300d\u4e86\u3002\u4f46\u9019\u53f0\u8ff7\u4f60 LSTM \u5b83\u5beb\u5b8c\u6559\u5ba4\u7684\u5287\u60c5\u4e4b\u5f8c\uff0cCell State\uff08\u8a18\u61b6\u9ad8\u901f\u516c\u8def\uff09\u6210\u529f\u7684\u628a\u300c\u8001\u738b\u9084\u6709\u53e6\u4e00\u500b\u6697\u5df7\u5287\u672c\u300d\u7684\u8a18\u61b6\u4e00\u8def\u50b3\u4e86\u4e0b\u4f86\uff0c\u6240\u4ee5\u5b83\u80fd\u5728\u5f8c\u534a\u53e5\u6210\u529f\u5207\u63db\u6642\u7a7a\uff0c\u958b\u59cb\u5beb\u300c\u8001\u738b\u5728\u6697\u5df7\u88e1\u9ede\u71c3\u4e86\u2026\u300d\u3002 \u4e86\u4e86\u4e86\u4e86\u4e86\uff1f \u90a3\u70ba\u4ec0\u9ebc\u5b83\u6703\u958b\u59cb\u50cf\u8df3\u91dd\u4e00\u6a23\u300c\u4e86\u4e86\u4e86\u4e86\u4e86\u4ed6\u300d\uff1f\u9019\u5728 NLP\uff08\u81ea\u7136\u8a9e\u8a00\u8655\u7406\uff09\u88e1\u53eb \u300c\u904e\u5ea6\u81ea\u4fe1\u8207\u8caa\u5a6a\u89e3\u6cd5\uff08Greedy Search\uff09\u7684\u7f3a\u9677\u300d\u3002 \u5728\u6211\u5011\u7684\u7a0b\u5f0f\u78bc\u4e2d\uff0c\u6709\u9019\u95dc\u9375\u7684\u4e00\u884c\uff1a \u56e0\u70ba\u6211\u5011\u7684\u53e5\u5b50\u88e1\u51fa\u73fe\u4e86\u975e\u5e38\u591a\u6b21\u7684\u300c\u4e86\u300d\uff08\u9ede\u71c3\u4e86\u3001\u7167\u4eae\u4e86\u3001\u62ff\u8d77\u4e86\u3001\u96e3\u5012\u4e86\uff09\u3002\u5c0d [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":520,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"footnotes":""},"class_list":["post-928","page","type-page","status-publish","hentry"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 4.9.10 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u5be6\u9a57\u4e00: RNN \u524d\u5f80\u4e0b\u8f09ai-lab\u7a0b\u5f0f\u78bc\uff0c\u5c07\u5176\u8cbc\u4e0agoogle colab\u9032\u884c\u5be6\u9a57! https:\/\/github.com\/Chen11111112\/ai-lab.git \u5be6\u9a571-1 - \u96b1\u85cf\u72c0\u614b \u5728\u505a\u4ec0\u9ebc \u7528\u4e00\u6bb5\u4e2d\u6587\u77ed\u6587\u8a13\u7df4\u4e00\u500b\u8ff7\u4f60 RNN\uff0c\u5b78\u7fd2\u300c\u7d66\u524d 3 \u500b\u5b57\uff0c\u731c\u7b2c 4 \u500b\u5b57\u300d\u3002 \u8a13\u7df4\u5b8c\u6210\u5f8c\uff0c\u7d66\u958b\u982d\u300c\u8001\u738b\u300d\uff0c\u8b93 AI \u81ea\u52d5\u7e8c\u5beb 20 \u500b\u5b57\u3002 \u6838\u5fc3\u6982\u5ff5 Embedding\uff1a\u628a\u5b57\u8f49\u6210\u5411\u91cf nn.RNN\uff1a\u4f9d\u5e8f\u8655\u7406\u6642\u9593\u6b65\uff0c\u7d2f\u7a4d\u96b1\u85cf\u72c0\u614b BPTT\uff08loss.backward()\uff09\uff1a\u6cbf\u6642\u9593\u53cd\u5411\u50b3\u64ad\uff0c\u66f4\u65b0\u6b0a\u91cd W optimizer.step()\uff1a\u6839\u64da\u68af\u5ea6\u8abf\u6574\u53c3\u6578 \u89c0\u5bdf\u91cd\u9ede \u7e8c\u5beb\u5167\u5bb9\u662f\u5426\u51fa\u73fe\u8a13\u7df4\u6587\u672c\u4e2d\u7684\u7247\u8a9e\uff08\u5982\u300c\u6697\u5df7\u300d\u300c\u6253\u706b\u6a5f\u300d\u300c\u5fae\u7a4d\u5206\u300d\uff09\uff1f \u6a21\u578b\u80fd\u5426\u5340\u5206\u5169\u689d\u6545\u4e8b\u7dda\uff08\u6697\u5df7 vs \u6559\u5ba4\uff09\uff1f \u5e8f\u5217\u9577\u5ea6\u50c5 3\uff0c\u6a21\u578b\u8a18\u61b6\u5f88\u77ed\uff0c\u5bb9\u6613\u300c\u5fd8\u8a18\u300d\u66f4\u65e9\u7684\u4e0a\u4e0b\u6587\u3002 \u591a\u57f7\u884c\u5e7e\u6b21\u6703\u767c\u73fe\u50b3\u7d71 RNN \u7684\u96b1\u85cf\u72c0\u614b\uff08Hidden State\uff09\u7121\u6cd5\u5728\u9577\u8ddd\u96e2\u7684\u6587\u672c\u4e2d\uff0c\u7cbe\u6e96\u5206\u914d\u300e\u73fe\u5728\u8a72\u628a\u6ce8\u610f\u529b\u653e\u5728\u6697\u5df7\u9084\u662f\u6559\u5ba4\u300f\u7684\u6b0a\u91cd\uff0c\u9019\u4e5f\u6210\u4e86\u5b83\u649e\u4e0a\u6280\u8853\u9ad8\u7246\u7684\u5be6\u8b49\uff0c\u9032\u800c\u63a8\u52d5\u4e86\u5f8c\u7e8c LSTM \u9580\u63a7\u6a5f\u5236 \u8207 Transformer \u6ce8\u610f\u529b\u6a5f\u5236 \u7684\u8a95\u751f\u3002 \u5be6\u9a571-2 - \u68af\u5ea6\u6d88\u5931 \u5728\u505a\u4ec0\u9ebc \u8207 1-1 \u76f8\u540c\u7684 RNN \u67b6\u69cb\uff0c\u4f46\u5c07 seq_length \u5f9e 3\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"google-site-verification\" content=\"-324YR5tomynX5FeTW2lTMHIX0KmPBvpxjXpOhXwi1Q\" \/>\n\t<link rel=\"canonical\" href=\"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 4.9.10\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"HyC Wiki - Technical articles\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"4. \u5be6\u9a57: \u624b\u523b NLP - HyC Wiki\" \/>\n\t\t<meta property=\"og:description\" content=\"\u5be6\u9a57\u4e00: RNN \u524d\u5f80\u4e0b\u8f09ai-lab\u7a0b\u5f0f\u78bc\uff0c\u5c07\u5176\u8cbc\u4e0agoogle colab\u9032\u884c\u5be6\u9a57! https:\/\/github.com\/Chen11111112\/ai-lab.git \u5be6\u9a571-1 - \u96b1\u85cf\u72c0\u614b \u5728\u505a\u4ec0\u9ebc \u7528\u4e00\u6bb5\u4e2d\u6587\u77ed\u6587\u8a13\u7df4\u4e00\u500b\u8ff7\u4f60 RNN\uff0c\u5b78\u7fd2\u300c\u7d66\u524d 3 \u500b\u5b57\uff0c\u731c\u7b2c 4 \u500b\u5b57\u300d\u3002 \u8a13\u7df4\u5b8c\u6210\u5f8c\uff0c\u7d66\u958b\u982d\u300c\u8001\u738b\u300d\uff0c\u8b93 AI \u81ea\u52d5\u7e8c\u5beb 20 \u500b\u5b57\u3002 \u6838\u5fc3\u6982\u5ff5 Embedding\uff1a\u628a\u5b57\u8f49\u6210\u5411\u91cf nn.RNN\uff1a\u4f9d\u5e8f\u8655\u7406\u6642\u9593\u6b65\uff0c\u7d2f\u7a4d\u96b1\u85cf\u72c0\u614b BPTT\uff08loss.backward()\uff09\uff1a\u6cbf\u6642\u9593\u53cd\u5411\u50b3\u64ad\uff0c\u66f4\u65b0\u6b0a\u91cd W optimizer.step()\uff1a\u6839\u64da\u68af\u5ea6\u8abf\u6574\u53c3\u6578 \u89c0\u5bdf\u91cd\u9ede \u7e8c\u5beb\u5167\u5bb9\u662f\u5426\u51fa\u73fe\u8a13\u7df4\u6587\u672c\u4e2d\u7684\u7247\u8a9e\uff08\u5982\u300c\u6697\u5df7\u300d\u300c\u6253\u706b\u6a5f\u300d\u300c\u5fae\u7a4d\u5206\u300d\uff09\uff1f \u6a21\u578b\u80fd\u5426\u5340\u5206\u5169\u689d\u6545\u4e8b\u7dda\uff08\u6697\u5df7 vs \u6559\u5ba4\uff09\uff1f \u5e8f\u5217\u9577\u5ea6\u50c5 3\uff0c\u6a21\u578b\u8a18\u61b6\u5f88\u77ed\uff0c\u5bb9\u6613\u300c\u5fd8\u8a18\u300d\u66f4\u65e9\u7684\u4e0a\u4e0b\u6587\u3002 \u591a\u57f7\u884c\u5e7e\u6b21\u6703\u767c\u73fe\u50b3\u7d71 RNN \u7684\u96b1\u85cf\u72c0\u614b\uff08Hidden State\uff09\u7121\u6cd5\u5728\u9577\u8ddd\u96e2\u7684\u6587\u672c\u4e2d\uff0c\u7cbe\u6e96\u5206\u914d\u300e\u73fe\u5728\u8a72\u628a\u6ce8\u610f\u529b\u653e\u5728\u6697\u5df7\u9084\u662f\u6559\u5ba4\u300f\u7684\u6b0a\u91cd\uff0c\u9019\u4e5f\u6210\u4e86\u5b83\u649e\u4e0a\u6280\u8853\u9ad8\u7246\u7684\u5be6\u8b49\uff0c\u9032\u800c\u63a8\u52d5\u4e86\u5f8c\u7e8c LSTM \u9580\u63a7\u6a5f\u5236 \u8207 Transformer \u6ce8\u610f\u529b\u6a5f\u5236 \u7684\u8a95\u751f\u3002 \u5be6\u9a571-2 - \u68af\u5ea6\u6d88\u5931 \u5728\u505a\u4ec0\u9ebc \u8207 1-1 \u76f8\u540c\u7684 RNN \u67b6\u69cb\uff0c\u4f46\u5c07 seq_length \u5f9e 3\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/06\/cropped-cropped-\u672a\u547d\u540d\u8a2d\u8a08-4.png\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/06\/cropped-cropped-\u672a\u547d\u540d\u8a2d\u8a08-4.png\" \/>\n\t\t<meta property=\"og:image:width\" content=\"64\" \/>\n\t\t<meta property=\"og:image:height\" content=\"64\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2026-07-04T05:15:57+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2026-07-04T06:13:16+00:00\" \/>\n\t\t<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n\t\t<meta name=\"twitter:title\" content=\"4. \u5be6\u9a57: \u624b\u523b NLP - HyC Wiki\" \/>\n\t\t<meta name=\"twitter:description\" content=\"\u5be6\u9a57\u4e00: RNN \u524d\u5f80\u4e0b\u8f09ai-lab\u7a0b\u5f0f\u78bc\uff0c\u5c07\u5176\u8cbc\u4e0agoogle colab\u9032\u884c\u5be6\u9a57! https:\/\/github.com\/Chen11111112\/ai-lab.git \u5be6\u9a571-1 - \u96b1\u85cf\u72c0\u614b \u5728\u505a\u4ec0\u9ebc \u7528\u4e00\u6bb5\u4e2d\u6587\u77ed\u6587\u8a13\u7df4\u4e00\u500b\u8ff7\u4f60 RNN\uff0c\u5b78\u7fd2\u300c\u7d66\u524d 3 \u500b\u5b57\uff0c\u731c\u7b2c 4 \u500b\u5b57\u300d\u3002 \u8a13\u7df4\u5b8c\u6210\u5f8c\uff0c\u7d66\u958b\u982d\u300c\u8001\u738b\u300d\uff0c\u8b93 AI \u81ea\u52d5\u7e8c\u5beb 20 \u500b\u5b57\u3002 \u6838\u5fc3\u6982\u5ff5 Embedding\uff1a\u628a\u5b57\u8f49\u6210\u5411\u91cf nn.RNN\uff1a\u4f9d\u5e8f\u8655\u7406\u6642\u9593\u6b65\uff0c\u7d2f\u7a4d\u96b1\u85cf\u72c0\u614b BPTT\uff08loss.backward()\uff09\uff1a\u6cbf\u6642\u9593\u53cd\u5411\u50b3\u64ad\uff0c\u66f4\u65b0\u6b0a\u91cd W optimizer.step()\uff1a\u6839\u64da\u68af\u5ea6\u8abf\u6574\u53c3\u6578 \u89c0\u5bdf\u91cd\u9ede \u7e8c\u5beb\u5167\u5bb9\u662f\u5426\u51fa\u73fe\u8a13\u7df4\u6587\u672c\u4e2d\u7684\u7247\u8a9e\uff08\u5982\u300c\u6697\u5df7\u300d\u300c\u6253\u706b\u6a5f\u300d\u300c\u5fae\u7a4d\u5206\u300d\uff09\uff1f \u6a21\u578b\u80fd\u5426\u5340\u5206\u5169\u689d\u6545\u4e8b\u7dda\uff08\u6697\u5df7 vs \u6559\u5ba4\uff09\uff1f \u5e8f\u5217\u9577\u5ea6\u50c5 3\uff0c\u6a21\u578b\u8a18\u61b6\u5f88\u77ed\uff0c\u5bb9\u6613\u300c\u5fd8\u8a18\u300d\u66f4\u65e9\u7684\u4e0a\u4e0b\u6587\u3002 \u591a\u57f7\u884c\u5e7e\u6b21\u6703\u767c\u73fe\u50b3\u7d71 RNN \u7684\u96b1\u85cf\u72c0\u614b\uff08Hidden State\uff09\u7121\u6cd5\u5728\u9577\u8ddd\u96e2\u7684\u6587\u672c\u4e2d\uff0c\u7cbe\u6e96\u5206\u914d\u300e\u73fe\u5728\u8a72\u628a\u6ce8\u610f\u529b\u653e\u5728\u6697\u5df7\u9084\u662f\u6559\u5ba4\u300f\u7684\u6b0a\u91cd\uff0c\u9019\u4e5f\u6210\u4e86\u5b83\u649e\u4e0a\u6280\u8853\u9ad8\u7246\u7684\u5be6\u8b49\uff0c\u9032\u800c\u63a8\u52d5\u4e86\u5f8c\u7e8c LSTM \u9580\u63a7\u6a5f\u5236 \u8207 Transformer \u6ce8\u610f\u529b\u6a5f\u5236 \u7684\u8a95\u751f\u3002 \u5be6\u9a571-2 - \u68af\u5ea6\u6d88\u5931 \u5728\u505a\u4ec0\u9ebc \u8207 1-1 \u76f8\u540c\u7684 RNN \u67b6\u69cb\uff0c\u4f46\u5c07 seq_length \u5f9e 3\" \/>\n\t\t<meta name=\"twitter:image\" content=\"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/06\/cropped-cropped-\u672a\u547d\u540d\u8a2d\u8a08-4.png\" \/>\n\t\t<script type=\"application\/ld+json\" class=\"aioseo-schema\">\n\t\t\t{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/llm-agent\\\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\\\/#breadcrumblist\",\"itemListElement\":[{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program#listItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/llm-agent\\\/#listItem\",\"name\":\"\\u3010RNN\\u5230Agent\\u3011\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/llm-agent\\\/#listItem\",\"position\":2,\"name\":\"\\u3010RNN\\u5230Agent\\u3011\",\"item\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/llm-agent\\\/\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/llm-agent\\\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\\\/#listItem\",\"name\":\"4. \\u5be6\\u9a57: \\u624b\\u523b NLP\"},\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program#listItem\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/llm-agent\\\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\\\/#listItem\",\"position\":3,\"name\":\"4. \\u5be6\\u9a57: \\u624b\\u523b NLP\",\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/llm-agent\\\/#listItem\",\"name\":\"\\u3010RNN\\u5230Agent\\u3011\"}}]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/#person\",\"name\":\"H.Y. Chen\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/llm-agent\\\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\\\/#personImage\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/fc6e05e080c9ea570267ee9ed68c8b1841d745cee07c0331c5440646e0646603?s=96&d=mm&r=g\",\"width\":96,\"height\":96,\"caption\":\"H.Y. Chen\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/llm-agent\\\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\\\/#webpage\",\"url\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/llm-agent\\\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\\\/\",\"name\":\"4. \\u5be6\\u9a57: \\u624b\\u523b NLP - HyC Wiki\",\"description\":\"\\u5be6\\u9a57\\u4e00: RNN \\u524d\\u5f80\\u4e0b\\u8f09ai-lab\\u7a0b\\u5f0f\\u78bc\\uff0c\\u5c07\\u5176\\u8cbc\\u4e0agoogle colab\\u9032\\u884c\\u5be6\\u9a57! https:\\\/\\\/github.com\\\/Chen11111112\\\/ai-lab.git \\u5be6\\u9a571-1 - \\u96b1\\u85cf\\u72c0\\u614b \\u5728\\u505a\\u4ec0\\u9ebc \\u7528\\u4e00\\u6bb5\\u4e2d\\u6587\\u77ed\\u6587\\u8a13\\u7df4\\u4e00\\u500b\\u8ff7\\u4f60 RNN\\uff0c\\u5b78\\u7fd2\\u300c\\u7d66\\u524d 3 \\u500b\\u5b57\\uff0c\\u731c\\u7b2c 4 \\u500b\\u5b57\\u300d\\u3002 \\u8a13\\u7df4\\u5b8c\\u6210\\u5f8c\\uff0c\\u7d66\\u958b\\u982d\\u300c\\u8001\\u738b\\u300d\\uff0c\\u8b93 AI \\u81ea\\u52d5\\u7e8c\\u5beb 20 \\u500b\\u5b57\\u3002 \\u6838\\u5fc3\\u6982\\u5ff5 Embedding\\uff1a\\u628a\\u5b57\\u8f49\\u6210\\u5411\\u91cf nn.RNN\\uff1a\\u4f9d\\u5e8f\\u8655\\u7406\\u6642\\u9593\\u6b65\\uff0c\\u7d2f\\u7a4d\\u96b1\\u85cf\\u72c0\\u614b BPTT\\uff08loss.backward()\\uff09\\uff1a\\u6cbf\\u6642\\u9593\\u53cd\\u5411\\u50b3\\u64ad\\uff0c\\u66f4\\u65b0\\u6b0a\\u91cd W optimizer.step()\\uff1a\\u6839\\u64da\\u68af\\u5ea6\\u8abf\\u6574\\u53c3\\u6578 \\u89c0\\u5bdf\\u91cd\\u9ede \\u7e8c\\u5beb\\u5167\\u5bb9\\u662f\\u5426\\u51fa\\u73fe\\u8a13\\u7df4\\u6587\\u672c\\u4e2d\\u7684\\u7247\\u8a9e\\uff08\\u5982\\u300c\\u6697\\u5df7\\u300d\\u300c\\u6253\\u706b\\u6a5f\\u300d\\u300c\\u5fae\\u7a4d\\u5206\\u300d\\uff09\\uff1f \\u6a21\\u578b\\u80fd\\u5426\\u5340\\u5206\\u5169\\u689d\\u6545\\u4e8b\\u7dda\\uff08\\u6697\\u5df7 vs \\u6559\\u5ba4\\uff09\\uff1f \\u5e8f\\u5217\\u9577\\u5ea6\\u50c5 3\\uff0c\\u6a21\\u578b\\u8a18\\u61b6\\u5f88\\u77ed\\uff0c\\u5bb9\\u6613\\u300c\\u5fd8\\u8a18\\u300d\\u66f4\\u65e9\\u7684\\u4e0a\\u4e0b\\u6587\\u3002 \\u591a\\u57f7\\u884c\\u5e7e\\u6b21\\u6703\\u767c\\u73fe\\u50b3\\u7d71 RNN \\u7684\\u96b1\\u85cf\\u72c0\\u614b\\uff08Hidden State\\uff09\\u7121\\u6cd5\\u5728\\u9577\\u8ddd\\u96e2\\u7684\\u6587\\u672c\\u4e2d\\uff0c\\u7cbe\\u6e96\\u5206\\u914d\\u300e\\u73fe\\u5728\\u8a72\\u628a\\u6ce8\\u610f\\u529b\\u653e\\u5728\\u6697\\u5df7\\u9084\\u662f\\u6559\\u5ba4\\u300f\\u7684\\u6b0a\\u91cd\\uff0c\\u9019\\u4e5f\\u6210\\u4e86\\u5b83\\u649e\\u4e0a\\u6280\\u8853\\u9ad8\\u7246\\u7684\\u5be6\\u8b49\\uff0c\\u9032\\u800c\\u63a8\\u52d5\\u4e86\\u5f8c\\u7e8c LSTM \\u9580\\u63a7\\u6a5f\\u5236 \\u8207 Transformer \\u6ce8\\u610f\\u529b\\u6a5f\\u5236 \\u7684\\u8a95\\u751f\\u3002 \\u5be6\\u9a571-2 - \\u68af\\u5ea6\\u6d88\\u5931 \\u5728\\u505a\\u4ec0\\u9ebc \\u8207 1-1 \\u76f8\\u540c\\u7684 RNN \\u67b6\\u69cb\\uff0c\\u4f46\\u5c07 seq_length \\u5f9e 3\",\"inLanguage\":\"en-US\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/#website\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/llm-agent\\\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\\\/#breadcrumblist\"},\"datePublished\":\"2026-07-04T05:15:57+00:00\",\"dateModified\":\"2026-07-04T06:13:16+00:00\"},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/#website\",\"url\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/\",\"name\":\"HyC Wiki\",\"description\":\"Technical articles\",\"inLanguage\":\"en-US\",\"publisher\":{\"@id\":\"https:\\\/\\\/hyc.eshachem.com\\\/program\\\/#person\"}}]}\n\t\t<\/script>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"4. \u5be6\u9a57: \u624b\u523b NLP - HyC Wiki","description":"\u5be6\u9a57\u4e00: RNN \u524d\u5f80\u4e0b\u8f09ai-lab\u7a0b\u5f0f\u78bc\uff0c\u5c07\u5176\u8cbc\u4e0agoogle colab\u9032\u884c\u5be6\u9a57! https:\/\/github.com\/Chen11111112\/ai-lab.git \u5be6\u9a571-1 - \u96b1\u85cf\u72c0\u614b \u5728\u505a\u4ec0\u9ebc \u7528\u4e00\u6bb5\u4e2d\u6587\u77ed\u6587\u8a13\u7df4\u4e00\u500b\u8ff7\u4f60 RNN\uff0c\u5b78\u7fd2\u300c\u7d66\u524d 3 \u500b\u5b57\uff0c\u731c\u7b2c 4 \u500b\u5b57\u300d\u3002 \u8a13\u7df4\u5b8c\u6210\u5f8c\uff0c\u7d66\u958b\u982d\u300c\u8001\u738b\u300d\uff0c\u8b93 AI \u81ea\u52d5\u7e8c\u5beb 20 \u500b\u5b57\u3002 \u6838\u5fc3\u6982\u5ff5 Embedding\uff1a\u628a\u5b57\u8f49\u6210\u5411\u91cf nn.RNN\uff1a\u4f9d\u5e8f\u8655\u7406\u6642\u9593\u6b65\uff0c\u7d2f\u7a4d\u96b1\u85cf\u72c0\u614b BPTT\uff08loss.backward()\uff09\uff1a\u6cbf\u6642\u9593\u53cd\u5411\u50b3\u64ad\uff0c\u66f4\u65b0\u6b0a\u91cd W optimizer.step()\uff1a\u6839\u64da\u68af\u5ea6\u8abf\u6574\u53c3\u6578 \u89c0\u5bdf\u91cd\u9ede \u7e8c\u5beb\u5167\u5bb9\u662f\u5426\u51fa\u73fe\u8a13\u7df4\u6587\u672c\u4e2d\u7684\u7247\u8a9e\uff08\u5982\u300c\u6697\u5df7\u300d\u300c\u6253\u706b\u6a5f\u300d\u300c\u5fae\u7a4d\u5206\u300d\uff09\uff1f \u6a21\u578b\u80fd\u5426\u5340\u5206\u5169\u689d\u6545\u4e8b\u7dda\uff08\u6697\u5df7 vs \u6559\u5ba4\uff09\uff1f \u5e8f\u5217\u9577\u5ea6\u50c5 3\uff0c\u6a21\u578b\u8a18\u61b6\u5f88\u77ed\uff0c\u5bb9\u6613\u300c\u5fd8\u8a18\u300d\u66f4\u65e9\u7684\u4e0a\u4e0b\u6587\u3002 \u591a\u57f7\u884c\u5e7e\u6b21\u6703\u767c\u73fe\u50b3\u7d71 RNN \u7684\u96b1\u85cf\u72c0\u614b\uff08Hidden State\uff09\u7121\u6cd5\u5728\u9577\u8ddd\u96e2\u7684\u6587\u672c\u4e2d\uff0c\u7cbe\u6e96\u5206\u914d\u300e\u73fe\u5728\u8a72\u628a\u6ce8\u610f\u529b\u653e\u5728\u6697\u5df7\u9084\u662f\u6559\u5ba4\u300f\u7684\u6b0a\u91cd\uff0c\u9019\u4e5f\u6210\u4e86\u5b83\u649e\u4e0a\u6280\u8853\u9ad8\u7246\u7684\u5be6\u8b49\uff0c\u9032\u800c\u63a8\u52d5\u4e86\u5f8c\u7e8c LSTM \u9580\u63a7\u6a5f\u5236 \u8207 Transformer \u6ce8\u610f\u529b\u6a5f\u5236 \u7684\u8a95\u751f\u3002 \u5be6\u9a571-2 - \u68af\u5ea6\u6d88\u5931 \u5728\u505a\u4ec0\u9ebc \u8207 1-1 \u76f8\u540c\u7684 RNN \u67b6\u69cb\uff0c\u4f46\u5c07 seq_length \u5f9e 3","canonical_url":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/","robots":"max-image-preview:large","keywords":"","webmasterTools":{"google-site-verification":"-324YR5tomynX5FeTW2lTMHIX0KmPBvpxjXpOhXwi1Q","miscellaneous":""},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"BreadcrumbList","@id":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/#breadcrumblist","itemListElement":[{"@type":"ListItem","@id":"https:\/\/hyc.eshachem.com\/program#listItem","position":1,"name":"Home","item":"https:\/\/hyc.eshachem.com\/program","nextItem":{"@type":"ListItem","@id":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/#listItem","name":"\u3010RNN\u5230Agent\u3011"}},{"@type":"ListItem","@id":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/#listItem","position":2,"name":"\u3010RNN\u5230Agent\u3011","item":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/","nextItem":{"@type":"ListItem","@id":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/#listItem","name":"4. \u5be6\u9a57: \u624b\u523b NLP"},"previousItem":{"@type":"ListItem","@id":"https:\/\/hyc.eshachem.com\/program#listItem","name":"Home"}},{"@type":"ListItem","@id":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/#listItem","position":3,"name":"4. \u5be6\u9a57: \u624b\u523b NLP","previousItem":{"@type":"ListItem","@id":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/#listItem","name":"\u3010RNN\u5230Agent\u3011"}}]},{"@type":"Person","@id":"https:\/\/hyc.eshachem.com\/program\/#person","name":"H.Y. Chen","image":{"@type":"ImageObject","@id":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/#personImage","url":"https:\/\/secure.gravatar.com\/avatar\/fc6e05e080c9ea570267ee9ed68c8b1841d745cee07c0331c5440646e0646603?s=96&d=mm&r=g","width":96,"height":96,"caption":"H.Y. Chen"}},{"@type":"WebPage","@id":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/#webpage","url":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/","name":"4. \u5be6\u9a57: \u624b\u523b NLP - HyC Wiki","description":"\u5be6\u9a57\u4e00: RNN \u524d\u5f80\u4e0b\u8f09ai-lab\u7a0b\u5f0f\u78bc\uff0c\u5c07\u5176\u8cbc\u4e0agoogle colab\u9032\u884c\u5be6\u9a57! https:\/\/github.com\/Chen11111112\/ai-lab.git \u5be6\u9a571-1 - \u96b1\u85cf\u72c0\u614b \u5728\u505a\u4ec0\u9ebc \u7528\u4e00\u6bb5\u4e2d\u6587\u77ed\u6587\u8a13\u7df4\u4e00\u500b\u8ff7\u4f60 RNN\uff0c\u5b78\u7fd2\u300c\u7d66\u524d 3 \u500b\u5b57\uff0c\u731c\u7b2c 4 \u500b\u5b57\u300d\u3002 \u8a13\u7df4\u5b8c\u6210\u5f8c\uff0c\u7d66\u958b\u982d\u300c\u8001\u738b\u300d\uff0c\u8b93 AI \u81ea\u52d5\u7e8c\u5beb 20 \u500b\u5b57\u3002 \u6838\u5fc3\u6982\u5ff5 Embedding\uff1a\u628a\u5b57\u8f49\u6210\u5411\u91cf nn.RNN\uff1a\u4f9d\u5e8f\u8655\u7406\u6642\u9593\u6b65\uff0c\u7d2f\u7a4d\u96b1\u85cf\u72c0\u614b BPTT\uff08loss.backward()\uff09\uff1a\u6cbf\u6642\u9593\u53cd\u5411\u50b3\u64ad\uff0c\u66f4\u65b0\u6b0a\u91cd W optimizer.step()\uff1a\u6839\u64da\u68af\u5ea6\u8abf\u6574\u53c3\u6578 \u89c0\u5bdf\u91cd\u9ede \u7e8c\u5beb\u5167\u5bb9\u662f\u5426\u51fa\u73fe\u8a13\u7df4\u6587\u672c\u4e2d\u7684\u7247\u8a9e\uff08\u5982\u300c\u6697\u5df7\u300d\u300c\u6253\u706b\u6a5f\u300d\u300c\u5fae\u7a4d\u5206\u300d\uff09\uff1f \u6a21\u578b\u80fd\u5426\u5340\u5206\u5169\u689d\u6545\u4e8b\u7dda\uff08\u6697\u5df7 vs \u6559\u5ba4\uff09\uff1f \u5e8f\u5217\u9577\u5ea6\u50c5 3\uff0c\u6a21\u578b\u8a18\u61b6\u5f88\u77ed\uff0c\u5bb9\u6613\u300c\u5fd8\u8a18\u300d\u66f4\u65e9\u7684\u4e0a\u4e0b\u6587\u3002 \u591a\u57f7\u884c\u5e7e\u6b21\u6703\u767c\u73fe\u50b3\u7d71 RNN \u7684\u96b1\u85cf\u72c0\u614b\uff08Hidden State\uff09\u7121\u6cd5\u5728\u9577\u8ddd\u96e2\u7684\u6587\u672c\u4e2d\uff0c\u7cbe\u6e96\u5206\u914d\u300e\u73fe\u5728\u8a72\u628a\u6ce8\u610f\u529b\u653e\u5728\u6697\u5df7\u9084\u662f\u6559\u5ba4\u300f\u7684\u6b0a\u91cd\uff0c\u9019\u4e5f\u6210\u4e86\u5b83\u649e\u4e0a\u6280\u8853\u9ad8\u7246\u7684\u5be6\u8b49\uff0c\u9032\u800c\u63a8\u52d5\u4e86\u5f8c\u7e8c LSTM \u9580\u63a7\u6a5f\u5236 \u8207 Transformer \u6ce8\u610f\u529b\u6a5f\u5236 \u7684\u8a95\u751f\u3002 \u5be6\u9a571-2 - \u68af\u5ea6\u6d88\u5931 \u5728\u505a\u4ec0\u9ebc \u8207 1-1 \u76f8\u540c\u7684 RNN \u67b6\u69cb\uff0c\u4f46\u5c07 seq_length \u5f9e 3","inLanguage":"en-US","isPartOf":{"@id":"https:\/\/hyc.eshachem.com\/program\/#website"},"breadcrumb":{"@id":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/#breadcrumblist"},"datePublished":"2026-07-04T05:15:57+00:00","dateModified":"2026-07-04T06:13:16+00:00"},{"@type":"WebSite","@id":"https:\/\/hyc.eshachem.com\/program\/#website","url":"https:\/\/hyc.eshachem.com\/program\/","name":"HyC Wiki","description":"Technical articles","inLanguage":"en-US","publisher":{"@id":"https:\/\/hyc.eshachem.com\/program\/#person"}}]},"og:locale":"en_US","og:site_name":"HyC Wiki - Technical articles","og:type":"article","og:title":"4. \u5be6\u9a57: \u624b\u523b NLP - HyC Wiki","og:description":"\u5be6\u9a57\u4e00: RNN \u524d\u5f80\u4e0b\u8f09ai-lab\u7a0b\u5f0f\u78bc\uff0c\u5c07\u5176\u8cbc\u4e0agoogle colab\u9032\u884c\u5be6\u9a57! https:\/\/github.com\/Chen11111112\/ai-lab.git \u5be6\u9a571-1 - \u96b1\u85cf\u72c0\u614b \u5728\u505a\u4ec0\u9ebc \u7528\u4e00\u6bb5\u4e2d\u6587\u77ed\u6587\u8a13\u7df4\u4e00\u500b\u8ff7\u4f60 RNN\uff0c\u5b78\u7fd2\u300c\u7d66\u524d 3 \u500b\u5b57\uff0c\u731c\u7b2c 4 \u500b\u5b57\u300d\u3002 \u8a13\u7df4\u5b8c\u6210\u5f8c\uff0c\u7d66\u958b\u982d\u300c\u8001\u738b\u300d\uff0c\u8b93 AI \u81ea\u52d5\u7e8c\u5beb 20 \u500b\u5b57\u3002 \u6838\u5fc3\u6982\u5ff5 Embedding\uff1a\u628a\u5b57\u8f49\u6210\u5411\u91cf nn.RNN\uff1a\u4f9d\u5e8f\u8655\u7406\u6642\u9593\u6b65\uff0c\u7d2f\u7a4d\u96b1\u85cf\u72c0\u614b BPTT\uff08loss.backward()\uff09\uff1a\u6cbf\u6642\u9593\u53cd\u5411\u50b3\u64ad\uff0c\u66f4\u65b0\u6b0a\u91cd W optimizer.step()\uff1a\u6839\u64da\u68af\u5ea6\u8abf\u6574\u53c3\u6578 \u89c0\u5bdf\u91cd\u9ede \u7e8c\u5beb\u5167\u5bb9\u662f\u5426\u51fa\u73fe\u8a13\u7df4\u6587\u672c\u4e2d\u7684\u7247\u8a9e\uff08\u5982\u300c\u6697\u5df7\u300d\u300c\u6253\u706b\u6a5f\u300d\u300c\u5fae\u7a4d\u5206\u300d\uff09\uff1f \u6a21\u578b\u80fd\u5426\u5340\u5206\u5169\u689d\u6545\u4e8b\u7dda\uff08\u6697\u5df7 vs \u6559\u5ba4\uff09\uff1f \u5e8f\u5217\u9577\u5ea6\u50c5 3\uff0c\u6a21\u578b\u8a18\u61b6\u5f88\u77ed\uff0c\u5bb9\u6613\u300c\u5fd8\u8a18\u300d\u66f4\u65e9\u7684\u4e0a\u4e0b\u6587\u3002 \u591a\u57f7\u884c\u5e7e\u6b21\u6703\u767c\u73fe\u50b3\u7d71 RNN \u7684\u96b1\u85cf\u72c0\u614b\uff08Hidden State\uff09\u7121\u6cd5\u5728\u9577\u8ddd\u96e2\u7684\u6587\u672c\u4e2d\uff0c\u7cbe\u6e96\u5206\u914d\u300e\u73fe\u5728\u8a72\u628a\u6ce8\u610f\u529b\u653e\u5728\u6697\u5df7\u9084\u662f\u6559\u5ba4\u300f\u7684\u6b0a\u91cd\uff0c\u9019\u4e5f\u6210\u4e86\u5b83\u649e\u4e0a\u6280\u8853\u9ad8\u7246\u7684\u5be6\u8b49\uff0c\u9032\u800c\u63a8\u52d5\u4e86\u5f8c\u7e8c LSTM \u9580\u63a7\u6a5f\u5236 \u8207 Transformer \u6ce8\u610f\u529b\u6a5f\u5236 \u7684\u8a95\u751f\u3002 \u5be6\u9a571-2 - \u68af\u5ea6\u6d88\u5931 \u5728\u505a\u4ec0\u9ebc \u8207 1-1 \u76f8\u540c\u7684 RNN \u67b6\u69cb\uff0c\u4f46\u5c07 seq_length \u5f9e 3","og:url":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/","og:image":"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/06\/cropped-cropped-\u672a\u547d\u540d\u8a2d\u8a08-4.png","og:image:secure_url":"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/06\/cropped-cropped-\u672a\u547d\u540d\u8a2d\u8a08-4.png","og:image:width":64,"og:image:height":64,"article:published_time":"2026-07-04T05:15:57+00:00","article:modified_time":"2026-07-04T06:13:16+00:00","twitter:card":"summary_large_image","twitter:title":"4. \u5be6\u9a57: \u624b\u523b NLP - HyC Wiki","twitter:description":"\u5be6\u9a57\u4e00: RNN \u524d\u5f80\u4e0b\u8f09ai-lab\u7a0b\u5f0f\u78bc\uff0c\u5c07\u5176\u8cbc\u4e0agoogle colab\u9032\u884c\u5be6\u9a57! https:\/\/github.com\/Chen11111112\/ai-lab.git \u5be6\u9a571-1 - \u96b1\u85cf\u72c0\u614b \u5728\u505a\u4ec0\u9ebc \u7528\u4e00\u6bb5\u4e2d\u6587\u77ed\u6587\u8a13\u7df4\u4e00\u500b\u8ff7\u4f60 RNN\uff0c\u5b78\u7fd2\u300c\u7d66\u524d 3 \u500b\u5b57\uff0c\u731c\u7b2c 4 \u500b\u5b57\u300d\u3002 \u8a13\u7df4\u5b8c\u6210\u5f8c\uff0c\u7d66\u958b\u982d\u300c\u8001\u738b\u300d\uff0c\u8b93 AI \u81ea\u52d5\u7e8c\u5beb 20 \u500b\u5b57\u3002 \u6838\u5fc3\u6982\u5ff5 Embedding\uff1a\u628a\u5b57\u8f49\u6210\u5411\u91cf nn.RNN\uff1a\u4f9d\u5e8f\u8655\u7406\u6642\u9593\u6b65\uff0c\u7d2f\u7a4d\u96b1\u85cf\u72c0\u614b BPTT\uff08loss.backward()\uff09\uff1a\u6cbf\u6642\u9593\u53cd\u5411\u50b3\u64ad\uff0c\u66f4\u65b0\u6b0a\u91cd W optimizer.step()\uff1a\u6839\u64da\u68af\u5ea6\u8abf\u6574\u53c3\u6578 \u89c0\u5bdf\u91cd\u9ede \u7e8c\u5beb\u5167\u5bb9\u662f\u5426\u51fa\u73fe\u8a13\u7df4\u6587\u672c\u4e2d\u7684\u7247\u8a9e\uff08\u5982\u300c\u6697\u5df7\u300d\u300c\u6253\u706b\u6a5f\u300d\u300c\u5fae\u7a4d\u5206\u300d\uff09\uff1f \u6a21\u578b\u80fd\u5426\u5340\u5206\u5169\u689d\u6545\u4e8b\u7dda\uff08\u6697\u5df7 vs \u6559\u5ba4\uff09\uff1f \u5e8f\u5217\u9577\u5ea6\u50c5 3\uff0c\u6a21\u578b\u8a18\u61b6\u5f88\u77ed\uff0c\u5bb9\u6613\u300c\u5fd8\u8a18\u300d\u66f4\u65e9\u7684\u4e0a\u4e0b\u6587\u3002 \u591a\u57f7\u884c\u5e7e\u6b21\u6703\u767c\u73fe\u50b3\u7d71 RNN \u7684\u96b1\u85cf\u72c0\u614b\uff08Hidden State\uff09\u7121\u6cd5\u5728\u9577\u8ddd\u96e2\u7684\u6587\u672c\u4e2d\uff0c\u7cbe\u6e96\u5206\u914d\u300e\u73fe\u5728\u8a72\u628a\u6ce8\u610f\u529b\u653e\u5728\u6697\u5df7\u9084\u662f\u6559\u5ba4\u300f\u7684\u6b0a\u91cd\uff0c\u9019\u4e5f\u6210\u4e86\u5b83\u649e\u4e0a\u6280\u8853\u9ad8\u7246\u7684\u5be6\u8b49\uff0c\u9032\u800c\u63a8\u52d5\u4e86\u5f8c\u7e8c LSTM \u9580\u63a7\u6a5f\u5236 \u8207 Transformer \u6ce8\u610f\u529b\u6a5f\u5236 \u7684\u8a95\u751f\u3002 \u5be6\u9a571-2 - \u68af\u5ea6\u6d88\u5931 \u5728\u505a\u4ec0\u9ebc \u8207 1-1 \u76f8\u540c\u7684 RNN \u67b6\u69cb\uff0c\u4f46\u5c07 seq_length \u5f9e 3","twitter:image":"https:\/\/hyc.eshachem.com\/program\/wp-content\/uploads\/2026\/06\/cropped-cropped-\u672a\u547d\u540d\u8a2d\u8a08-4.png"},"aioseo_meta_data":{"post_id":"928","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_custom_url":null,"og_image_custom_fields":null,"og_image_url":null,"og_image_width":null,"og_image_height":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_image_url":null,"twitter_title":null,"twitter_description":null,"schema_type":"default","schema_type_options":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"limit_modified_date":false,"ai":null,"breadcrumb_settings":null,"seo_analyzer_scan_date":null,"created":"2026-07-28 14:29:00","updated":"2026-07-28 14:29:00"},"aioseo_breadcrumb":"<div class=\"aioseo-breadcrumbs\"><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/hyc.eshachem.com\/program\" title=\"Home\">Home<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/hyc.eshachem.com\/program\/llm-agent\/\" title=\"\u3010RNN\u5230Agent\u3011\">\u3010RNN\u5230Agent\u3011<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\t4. \u5be6\u9a57: \u624b\u523b NLP\n\t\t<\/span><\/div>","aioseo_breadcrumb_json":[{"label":"Home","link":"https:\/\/hyc.eshachem.com\/program"},{"label":"\u3010RNN\u5230Agent\u3011","link":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/"},{"label":"4. \u5be6\u9a57: \u624b\u523b NLP","link":"https:\/\/hyc.eshachem.com\/program\/llm-agent\/4-%e5%af%a6%e9%a9%97-%e6%89%8b%e5%88%bb-nlp\/"}],"_links":{"self":[{"href":"https:\/\/hyc.eshachem.com\/program\/wp-json\/wp\/v2\/pages\/928","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hyc.eshachem.com\/program\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/hyc.eshachem.com\/program\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/hyc.eshachem.com\/program\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hyc.eshachem.com\/program\/wp-json\/wp\/v2\/comments?post=928"}],"version-history":[{"count":10,"href":"https:\/\/hyc.eshachem.com\/program\/wp-json\/wp\/v2\/pages\/928\/revisions"}],"predecessor-version":[{"id":947,"href":"https:\/\/hyc.eshachem.com\/program\/wp-json\/wp\/v2\/pages\/928\/revisions\/947"}],"up":[{"embeddable":true,"href":"https:\/\/hyc.eshachem.com\/program\/wp-json\/wp\/v2\/pages\/520"}],"wp:attachment":[{"href":"https:\/\/hyc.eshachem.com\/program\/wp-json\/wp\/v2\/media?parent=928"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}