Revealing the fog-of-war

A visualization-directed, uncertainty-aware approach for exploring high-dimensional data

Yang Wang, Kwan-Liu Ma

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Dimensionality Reduction (DR) is a crucial tool to facilitate high-dimensional data analysis. As the volume and the variety of features used to describe a phenomenon keeps increasing, DR has become not only desirable but paramount. However, DR can result in unreliable depictions of data. The uncertainties involved in DR may stem from the selection of methods, parameter configurations, and the constraints imposed by the user. To address these uncertainties, various means of DR quality assessment have been proposed in the literature. Nevertheless, how to optimize the trade-off between the quantification efficiency and accuracy is yet to be further studied. The purpose of this paper is to present a general technique, in the context of visual analytics, to support efficient uncertainty-aware high-dimensional data exploration. We model the uncertainty based on how well neighborhood geometries are preserved during DR. We employ approximated nearest neighbor (ANN) search algorithms to speed up the quantification process with marginal decrease in accuracy. We then visualize the quantified uncertainties in the form of augmented scatter plot. We test our technique with three real world datasets against several well-known DR techniques, and discuss possible underlying causes that lead to certain embedding patterns. Our results show that our approach is effective and beneficial for both DR assessment and user-centered data exploration.

Original languageEnglish (US)
Title of host publicationProceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015
EditorsFeng Luo, Kemafor Ogan, Mohammed J. Zaki, Laura Haas, Beng Chin Ooi, Vipin Kumar, Sudarsan Rachuri, Saumyadipta Pyne, Howard Ho, Xiaohua Hu, Shipeng Yu, Morris Hui-I Hsiao, Jian Li
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages629-638
Number of pages10
ISBN (Electronic)9781479999255
DOIs
StatePublished - Dec 22 2015
Event3rd IEEE International Conference on Big Data, IEEE Big Data 2015 - Santa Clara, United States
Duration: Oct 29 2015Nov 1 2015

Other

Other3rd IEEE International Conference on Big Data, IEEE Big Data 2015
CountryUnited States
CitySanta Clara
Period10/29/1511/1/15

Fingerprint

Fog
Visualization
Uncertainty
Geometry

Keywords

  • Dimension Reduction
  • Quality Assessment
  • Uncertainty Analysis
  • Visual Analytics

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Information Systems
  • Software

Cite this

Wang, Y., & Ma, K-L. (2015). Revealing the fog-of-war: A visualization-directed, uncertainty-aware approach for exploring high-dimensional data. In F. Luo, K. Ogan, M. J. Zaki, L. Haas, B. C. Ooi, V. Kumar, S. Rachuri, S. Pyne, H. Ho, X. Hu, S. Yu, M. H-I. Hsiao, ... J. Li (Eds.), Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015 (pp. 629-638). [7363807] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2015.7363807

Revealing the fog-of-war : A visualization-directed, uncertainty-aware approach for exploring high-dimensional data. / Wang, Yang; Ma, Kwan-Liu.

Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015. ed. / Feng Luo; Kemafor Ogan; Mohammed J. Zaki; Laura Haas; Beng Chin Ooi; Vipin Kumar; Sudarsan Rachuri; Saumyadipta Pyne; Howard Ho; Xiaohua Hu; Shipeng Yu; Morris Hui-I Hsiao; Jian Li. Institute of Electrical and Electronics Engineers Inc., 2015. p. 629-638 7363807.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, Y & Ma, K-L 2015, Revealing the fog-of-war: A visualization-directed, uncertainty-aware approach for exploring high-dimensional data. in F Luo, K Ogan, MJ Zaki, L Haas, BC Ooi, V Kumar, S Rachuri, S Pyne, H Ho, X Hu, S Yu, MH-I Hsiao & J Li (eds), Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015., 7363807, Institute of Electrical and Electronics Engineers Inc., pp. 629-638, 3rd IEEE International Conference on Big Data, IEEE Big Data 2015, Santa Clara, United States, 10/29/15. https://doi.org/10.1109/BigData.2015.7363807
Wang Y, Ma K-L. Revealing the fog-of-war: A visualization-directed, uncertainty-aware approach for exploring high-dimensional data. In Luo F, Ogan K, Zaki MJ, Haas L, Ooi BC, Kumar V, Rachuri S, Pyne S, Ho H, Hu X, Yu S, Hsiao MH-I, Li J, editors, Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 629-638. 7363807 https://doi.org/10.1109/BigData.2015.7363807
Wang, Yang ; Ma, Kwan-Liu. / Revealing the fog-of-war : A visualization-directed, uncertainty-aware approach for exploring high-dimensional data. Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015. editor / Feng Luo ; Kemafor Ogan ; Mohammed J. Zaki ; Laura Haas ; Beng Chin Ooi ; Vipin Kumar ; Sudarsan Rachuri ; Saumyadipta Pyne ; Howard Ho ; Xiaohua Hu ; Shipeng Yu ; Morris Hui-I Hsiao ; Jian Li. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 629-638
@inproceedings{1308c01e9ce44f199f9a71ebbba7a704,
title = "Revealing the fog-of-war: A visualization-directed, uncertainty-aware approach for exploring high-dimensional data",
abstract = "Dimensionality Reduction (DR) is a crucial tool to facilitate high-dimensional data analysis. As the volume and the variety of features used to describe a phenomenon keeps increasing, DR has become not only desirable but paramount. However, DR can result in unreliable depictions of data. The uncertainties involved in DR may stem from the selection of methods, parameter configurations, and the constraints imposed by the user. To address these uncertainties, various means of DR quality assessment have been proposed in the literature. Nevertheless, how to optimize the trade-off between the quantification efficiency and accuracy is yet to be further studied. The purpose of this paper is to present a general technique, in the context of visual analytics, to support efficient uncertainty-aware high-dimensional data exploration. We model the uncertainty based on how well neighborhood geometries are preserved during DR. We employ approximated nearest neighbor (ANN) search algorithms to speed up the quantification process with marginal decrease in accuracy. We then visualize the quantified uncertainties in the form of augmented scatter plot. We test our technique with three real world datasets against several well-known DR techniques, and discuss possible underlying causes that lead to certain embedding patterns. Our results show that our approach is effective and beneficial for both DR assessment and user-centered data exploration.",
keywords = "Dimension Reduction, Quality Assessment, Uncertainty Analysis, Visual Analytics",
author = "Yang Wang and Kwan-Liu Ma",
year = "2015",
month = "12",
day = "22",
doi = "10.1109/BigData.2015.7363807",
language = "English (US)",
pages = "629--638",
editor = "Feng Luo and Kemafor Ogan and Zaki, {Mohammed J.} and Laura Haas and Ooi, {Beng Chin} and Vipin Kumar and Sudarsan Rachuri and Saumyadipta Pyne and Howard Ho and Xiaohua Hu and Shipeng Yu and Hsiao, {Morris Hui-I} and Jian Li",
booktitle = "Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Revealing the fog-of-war

T2 - A visualization-directed, uncertainty-aware approach for exploring high-dimensional data

AU - Wang, Yang

AU - Ma, Kwan-Liu

PY - 2015/12/22

Y1 - 2015/12/22

N2 - Dimensionality Reduction (DR) is a crucial tool to facilitate high-dimensional data analysis. As the volume and the variety of features used to describe a phenomenon keeps increasing, DR has become not only desirable but paramount. However, DR can result in unreliable depictions of data. The uncertainties involved in DR may stem from the selection of methods, parameter configurations, and the constraints imposed by the user. To address these uncertainties, various means of DR quality assessment have been proposed in the literature. Nevertheless, how to optimize the trade-off between the quantification efficiency and accuracy is yet to be further studied. The purpose of this paper is to present a general technique, in the context of visual analytics, to support efficient uncertainty-aware high-dimensional data exploration. We model the uncertainty based on how well neighborhood geometries are preserved during DR. We employ approximated nearest neighbor (ANN) search algorithms to speed up the quantification process with marginal decrease in accuracy. We then visualize the quantified uncertainties in the form of augmented scatter plot. We test our technique with three real world datasets against several well-known DR techniques, and discuss possible underlying causes that lead to certain embedding patterns. Our results show that our approach is effective and beneficial for both DR assessment and user-centered data exploration.

AB - Dimensionality Reduction (DR) is a crucial tool to facilitate high-dimensional data analysis. As the volume and the variety of features used to describe a phenomenon keeps increasing, DR has become not only desirable but paramount. However, DR can result in unreliable depictions of data. The uncertainties involved in DR may stem from the selection of methods, parameter configurations, and the constraints imposed by the user. To address these uncertainties, various means of DR quality assessment have been proposed in the literature. Nevertheless, how to optimize the trade-off between the quantification efficiency and accuracy is yet to be further studied. The purpose of this paper is to present a general technique, in the context of visual analytics, to support efficient uncertainty-aware high-dimensional data exploration. We model the uncertainty based on how well neighborhood geometries are preserved during DR. We employ approximated nearest neighbor (ANN) search algorithms to speed up the quantification process with marginal decrease in accuracy. We then visualize the quantified uncertainties in the form of augmented scatter plot. We test our technique with three real world datasets against several well-known DR techniques, and discuss possible underlying causes that lead to certain embedding patterns. Our results show that our approach is effective and beneficial for both DR assessment and user-centered data exploration.

KW - Dimension Reduction

KW - Quality Assessment

KW - Uncertainty Analysis

KW - Visual Analytics

UR - http://www.scopus.com/inward/record.url?scp=84963761245&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84963761245&partnerID=8YFLogxK

U2 - 10.1109/BigData.2015.7363807

DO - 10.1109/BigData.2015.7363807

M3 - Conference contribution

SP - 629

EP - 638

BT - Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015

A2 - Luo, Feng

A2 - Ogan, Kemafor

A2 - Zaki, Mohammed J.

A2 - Haas, Laura

A2 - Ooi, Beng Chin

A2 - Kumar, Vipin

A2 - Rachuri, Sudarsan

A2 - Pyne, Saumyadipta

A2 - Ho, Howard

A2 - Hu, Xiaohua

A2 - Yu, Shipeng

A2 - Hsiao, Morris Hui-I

A2 - Li, Jian

PB - Institute of Electrical and Electronics Engineers Inc.

ER -