https://park.is/categories/nlp/Recent content in NLP on Ye Joo Park's BlogHugoen-USThu, 23 Jan 2025 00:00:00 +0000https://park.is/notebooks/20250123_sentiment_analysis_with_spacy_and_distilbert/Thu, 23 Jan 2025 00:00:00 +0000https://park.is/notebooks/20250123_sentiment_analysis_with_spacy_and_distilbert/<div class="cell border-box-sizing text_cell rendered" id="cell-id=0d6e4426-8fbd-4434-a2cf-32e7f6116cc3" > <div class="prompt input_prompt"></div> <div class="inner_cell"> <div class="text_cell_render border-box-sizing rendered_html"> <p> Natural language processing (NLP) aims to give computers the ability to understand, process, and even generate human language. This notebook introduces the common preprocessing steps and demonstrates how to use a widely used transformer model (<code>distilbert-base-uncased-finetuned-sst-2-english</code>) to perfrom a sentiment analysis. 😀😦🙁 </p> </div> </div> </div> <div class="cell border-box-sizing code_cell rendered" id="cell-id=a31b7a34-007c-4e89-8916-a07406598896" > <div class="input"> <div class="prompt input_prompt">In [1]:</div> <div class="inner_cell"> <div class="input_area"> <div class="highlight hl-ipython3"> <pre><span></span><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span> <span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span> <span class="kn">import</span> <span class="nn">plotly.express</span> <span class="k">as</span> <span class="nn">px</span> </pre> </div> </div> </div> </div> </div> <div class="cell border-box-sizing code_cell rendered" id="cell-id=8d48c8d9-b347-4596-afe2-acbe54df942c" > <div class="input"> <div class="prompt input_prompt">In [2]:</div> <div class="inner_cell"> <div class="input_area"> <div class="highlight hl-ipython3"> <pre><span></span><span class="n">pd</span><span class="o">.</span><span class="n">set_option</span><span class="p">(</span><span class="s1">'display.max_columns'</span><span class="p">,</span> <span class="mi">100</span><span class="p">)</span> </pre> </div> </div> </div> </div> </div> <div class="cell border-box-sizing text_cell rendered" id="cell-id=fa8fe7ba-291c-4abc-b95a-f80f4d6bba1f" > <div class="prompt input_prompt"></div> <div class="inner_cell"> <div class="text_cell_render border-box-sizing rendered_html"> <h2 id="%F0%9F%97%83%EF%B8%8F-Load-data"> 🗃️ Load data<a class="anchor-link" href="#%F0%9F%97%83%EF%B8%8F-Load-data" >¶</a > </h2> <p> This exercise uses a small dataset that contains reviews of two apartments at Indiana University Bloomington. </p>