Sentiment attributes(sentimentValue, sentiment) were added to token tag

[ramanqul]

example:

          <token id="1" sentimentValue="2" sentiment="Neutral">
            <word>This</word>
            <CharacterOffsetBegin>0</CharacterOffsetBegin>
            <CharacterOffsetEnd>4</CharacterOffsetEnd>
            <POS>DT</POS>
          </token>

[AngledLuffa]

A better way to accomplish this would be to have the SentimentAnnotator
attach the relevant information to the token. Perhaps one could even
debinarize the tree and attach the sentiment to each node of the original
tree.

John
On Jan 17, 2014 1:54 AM, "braman" [email protected] wrote:

example:

This
0
4

DT

You can merge this Pull Request by running

git pull https://github.com/braman/CoreNLP master

Or view, comment on, or merge it at:

#10
Commit Summary

• Sentiment attributes(sentimentValue, sentiment) were added to token
  tag,

File Changes

• M src/edu/stanford/nlp/pipeline/XMLOutputter.javahttps://github.com/Sentiment attributes(sentimentValue, sentiment) were added to token tag #10/files#diff-0(53)

Patch Links:

• https://github.com/stanfordnlp/CoreNLP/pull/10.patch
• https://github.com/stanfordnlp/CoreNLP/pull/10.diff

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/10
.

[ramanqul]

If you print the SentimentAnnotator three it will look this:

<ROOT score=1.0>
  <NP score=2.0>
    <DT score=2.0>
      <This score=NaN/>
    </DT>
    <NN score=2.0>
      <movie score=NaN/>
    </NN>
  </NP>
  <@S : NP ... score=1.0>
    <VP score=1.0>
      <@VP : ... VP score=1.0>
        <VBZ score=2.0>
          <does score=NaN/>
        </VBZ>
        <RB score=2.0>
          <n&apos;t score=NaN/>
        </RB>
      </@VP : ... VP>
      <VP score=2.0>
        <VB score=3.0>
          <care score=NaN/>
        </VB>
        <PP score=3.0>
          <IN score=2.0>
            <about score=NaN/>
          </IN>
          <NP score=3.0>
            <@NP : ... NP score=2.0>
              <NP score=3.0>
                <cleverness score=NaN/>
              </NP>
              <CC score=2.0>
                <or score=NaN/>
              </CC>
            </@NP : ... NP>
            <NP score=3.0>
              <NP score=2.0>
                <DT score=2.0>
                  <any score=NaN/>
                </DT>
                <@NP : DT ... score=2.0>
                  <JJ score=2.0>
                    <other score=NaN/>
                  </JJ>
                  <NN score=3.0>
                    <kind score=NaN/>
                  </NN>
                </@NP : DT ...>
              </NP>
              <PP score=3.0>
                <IN score=2.0>
                  <of score=NaN/>
                </IN>
                <NP score=3.0>
                  <JJ score=3.0>
                    <intelligent score=NaN/>
                  </JJ>
                  <NN score=4.0>
                    <humor score=NaN/>
                  </NN>
                </NP>
              </PP>
            </NP>
          </NP>
        </PP>
      </VP>
    </VP>
    <. score=2.0>
      <. score=NaN/>
    </.>
  </@S : NP ...>
</ROOT>

And tokens looks like this:

<root>
  <document>
    <sentences>
      <sentence id="1" sentimentValue="1" sentiment="Negative">
        <tokens>
          <token id="1">
            <word>This</word>
            <CharacterOffsetBegin>0</CharacterOffsetBegin>
            <CharacterOffsetEnd>4</CharacterOffsetEnd>
            <POS>DT</POS>
          </token>
          <token id="2">
            <word>movie</word>
            <CharacterOffsetBegin>5</CharacterOffsetBegin>
            <CharacterOffsetEnd>10</CharacterOffsetEnd>
            <POS>NN</POS>
          </token>
          <token id="3">
            <word>does</word>
            <CharacterOffsetBegin>11</CharacterOffsetBegin>
            <CharacterOffsetEnd>15</CharacterOffsetEnd>
            <POS>VBZ</POS>
          </token>
          <token id="4">
            <word>n't</word>
            <CharacterOffsetBegin>15</CharacterOffsetBegin>
            <CharacterOffsetEnd>18</CharacterOffsetEnd>
            <POS>RB</POS>
          </token>
          <token id="5">
            <word>care</word>
            <CharacterOffsetBegin>19</CharacterOffsetBegin>
            <CharacterOffsetEnd>23</CharacterOffsetEnd>
            <POS>VB</POS>
          </token>
          <token id="6">
            <word>about</word>
            <CharacterOffsetBegin>24</CharacterOffsetBegin>
            <CharacterOffsetEnd>29</CharacterOffsetEnd>
            <POS>IN</POS>
          </token>
          <token id="7">
            <word>cleverness</word>
            <CharacterOffsetBegin>30</CharacterOffsetBegin>
            <CharacterOffsetEnd>40</CharacterOffsetEnd>
            <POS>NNS</POS>
          </token>
          <token id="8">
            <word>or</word>
            <CharacterOffsetBegin>41</CharacterOffsetBegin>
            <CharacterOffsetEnd>43</CharacterOffsetEnd>
            <POS>CC</POS>
          </token>
          <token id="9">
            <word>any</word>
            <CharacterOffsetBegin>44</CharacterOffsetBegin>
            <CharacterOffsetEnd>47</CharacterOffsetEnd>
            <POS>DT</POS>
          </token>
          <token id="10">
            <word>other</word>
            <CharacterOffsetBegin>48</CharacterOffsetBegin>
            <CharacterOffsetEnd>53</CharacterOffsetEnd>
            <POS>JJ</POS>
          </token>
          <token id="11">
            <word>kind</word>
            <CharacterOffsetBegin>54</CharacterOffsetBegin>
            <CharacterOffsetEnd>58</CharacterOffsetEnd>
            <POS>NN</POS>
          </token>
          <token id="12">
            <word>of</word>
            <CharacterOffsetBegin>59</CharacterOffsetBegin>
            <CharacterOffsetEnd>61</CharacterOffsetEnd>
            <POS>IN</POS>
          </token>
          <token id="13">
            <word>intelligent</word>
            <CharacterOffsetBegin>62</CharacterOffsetBegin>
            <CharacterOffsetEnd>73</CharacterOffsetEnd>
            <POS>JJ</POS>
          </token>
          <token id="14">
            <word>humor</word>
            <CharacterOffsetBegin>74</CharacterOffsetBegin>
            <CharacterOffsetEnd>79</CharacterOffsetEnd>
            <POS>NN</POS>
          </token>
          <token id="15">
            <word>.</word>
            <CharacterOffsetBegin>79</CharacterOffsetBegin>
            <CharacterOffsetEnd>80</CharacterOffsetEnd>
            <POS>.</POS>
          </token>
        </tokens>
      </sentence>
    </sentences>
  </document>
</root>

As you can see above the only way to bind sentiments with tokens is to use token order id.
Also sentiment is a nested tree structure but the tokens are stored in list.

Ramankul

[AngledLuffa]

Perhaps what I meant was not correctly communicated. I meant that the
processing should be done in SentimentAnnotator and the printing in
XMLOutputter
On Jan 17, 2014 5:55 PM, "braman" [email protected] wrote:

If you print the SentimentAnnotator three it will look this:

<@s : NP ... score=1.0> <@vp : ... VP score=1.0> <@np : ... NP score=2.0>

<@np : DT ... score=2.0> <. score=2.0> <. score=NaN/>

And tokens looks like this:

This 0 4 DT movie 5 10 NN does 11 15 VBZ n't 15 18 RB care 19 23 VB about 24 29 IN cleverness 30 40 NNS or 41 43 CC any 44 47 DT other 48 53 JJ kind 54 58 NN of 59 61 IN intelligent 62 73 JJ humor 74 79 NN . 79 80 .

As you can see above the only way to bind sentiments with tokens is to use
token order id.
Also sentiment is a nested tree structure but the tokens are stored in
list.

Ramankul

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/10#issuecomment-32671587
.

[ramanqul]

Agreed, but how?
Could you show me an example, as I understand you want to do the processing in SentimentAnnotator.annotate method?

Regards

[AngledLuffa]

I mean that in the SentimentAnnotator you would attach the sentiment labels
to the tokens in the TokensAnnotation for each CoreMap. You could also
take the TreeAnnotation and for each span associate the sentiment labels
from the corresponding span from the binarized sentiment tree.
On Jan 18, 2014 7:30 AM, "braman" [email protected] wrote:

Agreed, but how?
Could you show me an example, as I understand you want to do the
processing in SentimentAnnotator.annotate method?

Regards

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/10#issuecomment-32684419
.

[manning]

Belatedly, I implemented a version of this, along the lines John suggested: The SentimentAnnotator puts sentiment values into tree nodes, and at present the outputters output the sentiment of each word token.