Nguyen Le Thanh, Hyunhwan “Aiden” Lee, Joseph Johnson, Mitsunori Ogihara, Gang Ren, and James W. Beauchamp, “Multi-Scale Auralization for Multimedia Analytical Feature Interaction”
2019 Audio Engineering Society (AES) 147th Pro Audio Convention in New York, NY.
Modern human-computer interaction systems use multiple perceptual dimensions to enhance intuition and efficiency of the user by improving their situational awareness. A signal processing and interaction framework is proposed for auralizing signal patterns for augmenting the visualization-focused analysis tasks of social media content analysis and annotations, with the goal of assisting the user in analyzing, retrieving, and organizing relevant information for marketing research. Audio signals are generated from video/audio signal patterns as an auralization framework, for example, using the audio frequency modulation that follows the magnitude contours of video color saturation. The integration of visual and aural presentations will benefit the user interactions by reducing the fatigue level and sharping the users’ sensitivity, thereby improving work efficiency, confidence, and satisfaction.
Hyunhwan “Aiden” Lee, Joseph Johnson, and Gerard J. Tellis. “Geo-Influence: Modeling Location-Specific Effects of Social Influence on Brand Preferences”
Revising manuscript for Journal of Marketing Research
The 41st ISMS Marketing Science Conference, University of Rome. June 20 – 22, 2019
The 2019 Haring Symposium, Indiana University. April 19 – 20, 2019
The 1st Interdisciplinary Research Cluster Day, University of Miami. April 1, 2019
The 3rd Annual Three Minute Thesis Competition, University of Miami. February 6, 2019
Behavioral Insights from Text Conference, University of Pennsylvania. January 17, 2020
Artificial Intelligence in Management Conference, University of Southern California. March 14, 2020
No brand is the market leader uniformly across geographic locations. Brands have strong and weak markets. Therefore, analysts need to track brand preferences by location. However, current methods of tracking geographic brand preferences (e.g., surveys, company sales, scanner panel data, Google Analytics, etc.) have limitations: limited sampling of brands, stores, and categories with time and region aggregation. We address these limitations through a spatial, nonparametric approach to track brand preferences at the micro-temporal and micro-geographic level using Twitter. Our approach inherits the intrinsic strengths of social media: micro-temporal, micro-geographic, faster than news media, higher frequency than sales, low cost, competitive monitoring, sentiment inclusion, and social network inclusion. A key feature of our approach is modeling the temporal geographic influence of social connections based on the followers of each post. We estimate our model for carbonated soft drinks brands. Our validation uses Pepsi sales, an apparel fashion brand, and the 2016 U.S. presidential election (a domain where marketing is critical). We find that geographic influence adds predictive power to micro-temporal and micro-geographic brand measures, leads online sales and predicts elections. Our study implies that the micro-temporal and micro-geographic social media measures provide actionable insights for marketing brands and political candidates.
The rapid advances in e-commerce and Web 2.0 technologies have greatly increased the impact of commercial advertisements on the general public. As a key enabling technology, a multitude of recommender systems exists which analyzes user features and browsing patterns to recommend appealing advertisements to users. In this work, we seek to study the characteristics or attributes that characterize an effective advertisement and recommend a useful set of features to aid the designing and production processes of commercial advertisements. We analyze the temporal patterns from multimedia content of advertisement videos including auditory, visual and textual components, and study their individual roles and synergies in the success of an advertisement. The objective of this work is then to measure the effectiveness of an advertisement, and to recommend a useful set of features to advertisement designers to make it more successful and approachable to users. Our proposed framework employs the signal processing technique of cross-modality feature learning where data streams from different components are employed to train separate neural network models and are then fused together to learn a shared representation. Subsequently, a neural network model trained on this joint feature embedding representation is utilized as a classifier to predict advertisement effectiveness. We validate our approach using subjective ratings from a dedicated user study, the sentiment strength of online viewer comments, and a viewer opinion metric of the ratio of the Likes and Views received by each advertisement from an online platform.
Gang Ren, Joseph Johnson, Hyunhwan “Aiden” Lee, Mitsunori
Ogihara. “Sequential Pattern Based Temporal Contour Representations for
Content-Based Multimedia Timeline Analysis”
2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), Anaheim, CA. pp. 657-664.
Temporal contour shapes are closely linked to the narrative structure of multimedia content and provide important reference points in content-based multimedia timeline analysis. In this paper, multimedia timeline is extracted from content as time varying video and audio signal features. A temporal contour representation is implemented based on sequential pattern discovery algorithm for modeling the variation contours of multimedia features. The proposed contour representation extracts repetitive temporal patterns from a hierarchy of time resolutions or from synchronized video/audio feature dimensions. The statistically significant contour components, depicting the dominant timeline shapes, are utilized as a structural or analytical representation of the timeline. The modeling performance of this proposed temporal modeling framework is demonstrated through empirical validation and subjective evaluations.
