Improving Downstream Task Performance by Treating Numbers as Entities

Dhanasekar Sundararaman; Vivek Subramanian; Guoyin Wang; Liyan Xu; Lawrence Carin

doi:10.1145/3511808.3557614

Improving Downstream Task Performance by Treating Numbers as Entities

Dhanasekar Sundararaman, Vivek Subramanian, Guoyin Wang, Liyan Xu, Lawrence Carin

Computer, Electrical and Mathematical Sciences and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed. Though numbers are typically not accounted for distinctly in most NLP tasks, there is still an underlying amount of numeracy already exhibited by NLP models. For instance, in named entity recognition (NER), numbers are not treated as an entity with distinct tags. In this work, we attempt to tap the potential of state-of-the-art language models and transfer their ability to boost performance in related downstream tasks dealing with numbers. Our proposed classification of numbers into entities helps NLP models perform well on several tasks, including a handcrafted Fill-In-The-Blank (FITB) task and on question answering, using joint embeddings, outperforming the BERT and RoBERTa baseline classification.

Original language	English (US)
Title of host publication	CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management
Publisher	Association for Computing Machinery
Pages	4535-4539
Number of pages	5
ISBN (Electronic)	9781450392365
DOIs	https://doi.org/10.1145/3511808.3557614
State	Published - Oct 17 2022
Event	31st ACM International Conference on Information and Knowledge Management, CIKM 2022 - Atlanta, United States Duration: Oct 17 2022 → Oct 21 2022

Publication series

Name	International Conference on Information and Knowledge Management, Proceedings

Conference

Conference	31st ACM International Conference on Information and Knowledge Management, CIKM 2022
Country/Territory	United States
City	Atlanta
Period	10/17/22 → 10/21/22

Bibliographical note

Publisher Copyright:
© 2022 ACM.

Keywords

named entity
numeracy
text tagging

ASJC Scopus subject areas

General Business, Management and Accounting
General Decision Sciences

Access to Document

10.1145/3511808.3557614

Cite this

Sundararaman, D., Subramanian, V., Wang, G., Xu, L., & Carin, L. (2022). Improving Downstream Task Performance by Treating Numbers as Entities. In CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management (pp. 4535-4539). (International Conference on Information and Knowledge Management, Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3511808.3557614

Sundararaman, Dhanasekar ; Subramanian, Vivek ; Wang, Guoyin et al. / Improving Downstream Task Performance by Treating Numbers as Entities. CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2022. pp. 4535-4539 (International Conference on Information and Knowledge Management, Proceedings).

@inproceedings{45fa8818fd31476581e28e83c4b3afbd,

title = "Improving Downstream Task Performance by Treating Numbers as Entities",

abstract = "Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed. Though numbers are typically not accounted for distinctly in most NLP tasks, there is still an underlying amount of numeracy already exhibited by NLP models. For instance, in named entity recognition (NER), numbers are not treated as an entity with distinct tags. In this work, we attempt to tap the potential of state-of-the-art language models and transfer their ability to boost performance in related downstream tasks dealing with numbers. Our proposed classification of numbers into entities helps NLP models perform well on several tasks, including a handcrafted Fill-In-The-Blank (FITB) task and on question answering, using joint embeddings, outperforming the BERT and RoBERTa baseline classification.",

keywords = "named entity, numeracy, text tagging",

author = "Dhanasekar Sundararaman and Vivek Subramanian and Guoyin Wang and Liyan Xu and Lawrence Carin",

note = "Publisher Copyright: {\textcopyright} 2022 ACM.; 31st ACM International Conference on Information and Knowledge Management, CIKM 2022 ; Conference date: 17-10-2022 Through 21-10-2022",

year = "2022",

month = oct,

day = "17",

doi = "10.1145/3511808.3557614",

language = "English (US)",

series = "International Conference on Information and Knowledge Management, Proceedings",

publisher = "Association for Computing Machinery",

pages = "4535--4539",

booktitle = "CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management",

}

Sundararaman, D, Subramanian, V, Wang, G, Xu, L & Carin, L 2022, Improving Downstream Task Performance by Treating Numbers as Entities. in CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management. International Conference on Information and Knowledge Management, Proceedings, Association for Computing Machinery, pp. 4535-4539, 31st ACM International Conference on Information and Knowledge Management, CIKM 2022, Atlanta, United States, 10/17/22. https://doi.org/10.1145/3511808.3557614

Improving Downstream Task Performance by Treating Numbers as Entities. / Sundararaman, Dhanasekar; Subramanian, Vivek; Wang, Guoyin et al.
CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2022. p. 4535-4539 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Improving Downstream Task Performance by Treating Numbers as Entities

AU - Sundararaman, Dhanasekar

AU - Subramanian, Vivek

AU - Wang, Guoyin

AU - Xu, Liyan

AU - Carin, Lawrence

PY - 2022/10/17

Y1 - 2022/10/17

N2 - Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed. Though numbers are typically not accounted for distinctly in most NLP tasks, there is still an underlying amount of numeracy already exhibited by NLP models. For instance, in named entity recognition (NER), numbers are not treated as an entity with distinct tags. In this work, we attempt to tap the potential of state-of-the-art language models and transfer their ability to boost performance in related downstream tasks dealing with numbers. Our proposed classification of numbers into entities helps NLP models perform well on several tasks, including a handcrafted Fill-In-The-Blank (FITB) task and on question answering, using joint embeddings, outperforming the BERT and RoBERTa baseline classification.

AB - Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed. Though numbers are typically not accounted for distinctly in most NLP tasks, there is still an underlying amount of numeracy already exhibited by NLP models. For instance, in named entity recognition (NER), numbers are not treated as an entity with distinct tags. In this work, we attempt to tap the potential of state-of-the-art language models and transfer their ability to boost performance in related downstream tasks dealing with numbers. Our proposed classification of numbers into entities helps NLP models perform well on several tasks, including a handcrafted Fill-In-The-Blank (FITB) task and on question answering, using joint embeddings, outperforming the BERT and RoBERTa baseline classification.

KW - named entity

KW - numeracy

KW - text tagging

UR - http://www.scopus.com/inward/record.url?scp=85140822954&partnerID=8YFLogxK

U2 - 10.1145/3511808.3557614

DO - 10.1145/3511808.3557614

M3 - Conference contribution

AN - SCOPUS:85140822954

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 4535

EP - 4539

BT - CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management

PB - Association for Computing Machinery

T2 - 31st ACM International Conference on Information and Knowledge Management, CIKM 2022

Y2 - 17 October 2022 through 21 October 2022

ER -

Sundararaman D, Subramanian V, Wang G, Xu L, Carin L. Improving Downstream Task Performance by Treating Numbers as Entities. In CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management. Association for Computing Machinery. 2022. p. 4535-4539. (International Conference on Information and Knowledge Management, Proceedings). doi: 10.1145/3511808.3557614

Improving Downstream Task Performance by Treating Numbers as Entities

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this