Screen Shot 2015-08-30 at 9.48.29 PM

Rhapsody Magazine

When the Web goes visual, it may just take
language right along with it
SEE WHAT I MEAN?

by Douglas Rushkoff,  December, 2015

These days, most smartphone keyboards dedicate more resources to emojis than to traditional alphanumeric characters. This development allows the iGeneration to communicate almost exclusively with emoticons and an endless array of seemingly meaningless images, such as identical twin girls wearing bunny ears. One winking smiley face says more for these kids than any message using the Roman alphabet could.

While it can be tempting to lament this development as the decline of written language in the Digital Age, I wonder if we should rather be appreciating emojis, animated GIFs, and the Web’s endless stream of memes as the tip of the iceberg for a coming paradigm shift in how we will communicate with each other. What if those little yellow faces are harbingers of an entirely new visual language?

Remember, every new medium starts off imitating its predecessors until it finds its own form. Early books were written versions of spoken stories. The first TV shows were staged plays performed in front of a camera. And now the Internet is using text, audio, and video as its content while it develops its own.

Indeed, as Web technology has gotten more sophisticated, it has moved steadily away from recreating that earlier media and toward more immersive experiences, like the Oculus Rift VR goggles or the planet-size interactive world of Minecraft. Just as emojis took over instant-messaging platforms, what language will rise to the occasion of 3-D virtual space?

Whatever it is it would have to be real-time, emotive, and universal—as iconic as emoticons yet as dynamic as zooming in and out on a Google map. It can’t just be a vocabulary the way emojis are. Such a new language will have to discover a syntax all its own, capable of expressing not just simple emotions but industrial information like medical data and stock market movements—everything we now do with text and graphics.

“Just as emojis took over instant-messaging platforms,
what language will rise to the occasion of 3-D virtual space?”

From what I’ve been able to gather from my friends at Facebook, Google, and Twitter, every major Silicon Valley player is racing to be the first to own such a system. One firm piling up such patents, Knowledge Visualizaton Systems, originally made its reputation creating interactive visualizations for better understanding brain trauma and climate change. KVS founder Alan Yelsey wants to extend such data visualization into what he calls “dynamic semiotics”—tools that can be used to express ideas and emotions in addition to information.

KVS is investing in research into how shapes and colors convey meaning and if those meanings are universal. Does red always mean danger? Do curves always suggest something organic? Can they translate those universal associations into a visual language that’s flexible enough to grow and spread on its own while remaining objective enough for business and science? If they can, then someday communication could well involve us donning goggles and painting scenes for one another by waving our arms—joining together in an abstract virtual universe. Stock traders will not just see how the NASDAQ is performing, they’ll also be able to share feelings about the numbers and gauge the collective mood. Medical students will not only watch visualizations of cancer as a biological phenomenon, they will also “see” its psychological impact on patients.

This notion of a shared visual landscape—a place where we can express ourselves through colors, shapes, and vibrating emergent forms—may sound like science fiction. Then again, the notion that the whole world would be communicating through handheld touchscreen computers was unimaginable just a couple of decades ago.

The idea that this new medium should get its own language is not just possible; it’s inevitable.

Douglas Rushkoff is a professor of digital economics, a world renowned speaker on technology and change, and a prolific author of such books at Present Shock: When Everything Happens Now. His forthcoming powerhouse book,  Throwing Rocks at the Google Bus: How Growth Became the Enemy of Prosperity, will be out shortly.

 Screen Shot 2015-08-30 at 10.05.20 PM

THE COMMUNICATING COMPLEXITY PROJECT

The Creation of a Universal Dynamic Semiotic Ontological Pattern Language Capable of Best Signifying Systemic Meaning

 

The Proposition: Those who study cognitive semiotics and linguistics, language, education, computer sciences, art and design, social sciences, neurology and other fields will participate in the cooperative development of a universal computer aided semiotic language that synthesizes and signifies systemics. The involvement of media/cognitive/educational and coding professionals to develop a modern semiotic language capable of processing complexity is a natural and critical evolution. Like the stages of semiotic development in the first 3 years of life, we humans are just at the early stages of our journey, and our actualization of a language for systemics will similarly begin with intentional systemic communications and the onset of various forms of dynamic systemic semiotic communication, leading to a fully realized multimedia narrative form capable of extracting, organizing, processing, synthesizing, storing, retrieving and communicating a higher level multi-variate narrative.

 

Only recently have we realized the software technology, real-time processing capacity and paradigm shifting potential to conceive and create a new language that provides the engine, tools and processes to effectively communicate complexity, systemics, multi-variate time/space relationships and the overwhelming progression of real time information and real time problems.

 

We believe the world needs a language that can tame complexity. Fields such as cognitive semiotics, linguistics, neurology, educational psychology, computer sciences and media communications are called upon to work together to address one of the greatest barriers to human progress and beneficent action – the lack of a universal friction free robust cognitively optimized language composed of synthesized systemic knowledge represented by intuitive dynamic semiotic forms that can be introduced and applied by anyone, regardless of inherited languages, cultures or educational levels.

 

We now endeavor to establish and fund a global project to apply dynamic semiotics and linguistics toward the invention, test and ultimate introduction of a new, robust, scalable semiotic language, toolset and systemic that tames the beast – complexity.

 

semiotic-models copy 2

 

 

Hypotheses to test:

 

1) Computer software, hardware and connectivity have reached a threshold that enables humans to utilize dynamic semiotic ontological worlds to represent meaning and objectives, processes and multi-variate relationships, knowledge and perspectives, proofs and uncertainties and participative and inclusive mediated content by using cognitively effective models, worlds, structures, forms and patterns that signify the highest levels of complex knowledge.

2) A dynamic real time generatively coded semiotic ontological modelset would add substantially to systemic knowledge accumulation and synthesis, comprehension, retention and effective multi-variate decision-making.

3) The genetic programming and wiring of human beings, coupled with the interplay among the conscious/subconscious/corporal, can be maximally inspired by an original, creative, appealing, computer aided semiotic world composed in such a way as to be culturally and cognitively accessible (through carefully designed semiotic forms and functions) to high value meaning linked to user objectives and knowledge scaffolds.

Background:

Human beings understand their world through the lenses of their inherited programming, their constructed languages, their sensory capacities, their manufactured communications mechanisms, their life experiences and their conscious interpretations and creative manipulations of all of the above.

From the beginning of human evolution until now, humans have adapted and evolved without the benefit of a systemic lens that would enable them to consciously perceive, test and understand the “what is” of the universe as the dynamic multivariate, multi-dimensional, multi-layered, multi-unknown construct that it is. Our existing forms of mass communication rely heavily on written and spoken primary symbols, (words and their accompanying sounds), inherited through centuries of unfocused cultural iteration. This form of language uses words as narrow building blocks that cannot represent a whole view or thought until all of the blocks are in place, and even then, once the wall of words is viewed or heard, the user is limited to a one or two dimensional form incapable of representing complexity, exchanges, interactions, relationships, proofs, alternative views and the inherent inadequacy of current knowledge – ever changing across time, space, state and levels of detail.

For the purposes of this concept paper outline, we are going to characterize human beings as a mutually inclusive dynamic compilation of conscious, subconscious and corporal constructs. This is a systemically informed choice, encouraging the reader to understand a human being as an exchange of matter and energy and bio/chemo/physio functions (corporal), autonomic/genetic programming and hidden inheritances (subconscious), and retained or real time self aware thought processes (consciousness).

Humans are a product of their genetic/inherited programming, their life experience, and their conscious, subconscious and corporal interaction with their life experience. Among the inheritances include a strong predisposition to identify deviations from the anticipated pattern of an environment or situation – across all sensory modalities. Inherited sight and sound processing tends to relegate the common, the every day, the expected, into general background noise, and to promote certain characteristics of sight and sound into immediate and important signal awareness. Unexpected movements and sounds, bright moving colors and lights, sharp piercing noises and specific music constructions, facial features with normative disfigurations or beauteous shapes, deeply signifying gestures and voice configurations, large vs small, loud vs soft, bold vs passive, red vs grey all produce wired human attention and memory because these stimuli all deliver meaning in the form of pattern.

Languages to interact with computers have successfully adopted some advancements in semiotics while languages among humans have remained relatively stable.

The languages humans use to communicate with computing devices have radically progressed. The languages humans use to communicate among each other, and within, have barely changed in over 200 exponential years of realized complexity. Human languages retain the barriers of literacy and manifold distinct versions and manifestations. Since 1970, computing languages have progressed in sophistication to the point that users can interact intuitively with computers without code, and the people that create languages that translate computer functions into desired functions use macro and meta languages, some with visual connectors and symbology, that subsume prior code and exponentially expand the power, prescience and functional capacity and diversity of computer interaction.

At the computer programming level, analogous to the human programming modes of genetic, experiential and self-programming, early laborious linear one- character-at-a-time machine languages have been superseded in practice by a small number of macro/meta languages that use simple forms of systemic representation to command the computer to interact with the coders and end users toward complexity, objectives and sophisticated multi-variate relationships and exchanges of content, form and function. Simple code sets, see Wolfram’s thesis on the deconstruction of everything, can produce generative complexities that “become alive”.

The capacity of human beings to construct meaning through dynamic and passive semiotic representations: symbols, pictures, ceremonies, letters/words/language, expressions, sounds, audio shapes and startles, motion startles-brightness-speed, clothing, jewelry, decorations, performance, art, colors, objects and structures is inextricably linked to our genetic programming which amplifies certain forms and patterns of these media so they become signals rather than noise. Our inherited life experiences enforce and mediate these representations of meaning – making any paradigm shifts in the scale, scope or mechanisms of these meaning conveyances difficult across culture, geography, time and institutional education. Once we inherit a language and linguistics, learn them and use them, they become more and more embedded as code, making it difficult for any new form to disrupt the status quo – what is.

We must remember that all of these forms of cognitive and creative representation were developed locally and incrementally, without explicit objectives, developmental architectures, academic foundations or functional optimizations. The constructs we call “language” and “linguistics” evolved incrementally, locally, without any significant master designer or planner laying out the most effective representations of meaning.

And at this moment, I am communicating to you, the reader, using primitive building block symbols and symbol sets that form learned meanings. There is nothing purely intuitive about this form of communication except for the fact that our sensory and cognitive functionalities enable us to seamlessly process these shapes and focus in on derived meaning, just like when watching television or movies we are transported, integrated, into the media, oblivious to what is around us. Here, the letters become noise, and the totality of shapes and context deliver the signal – autonomically.

Each of us is tuned to a certain spectrum of sensory channels, portals linking ourselves with our world, and ourselves, through which we more strongly perceive, receive, process as signal, store and utilize stimuli. We consciously, subconsciously and even corporally process the meaning and context of everything through our language and all forms of embedded representations of meaning. We think, believe, gut react and communicate within and without according to our language and representations.

The only way to intercept, redirect, disrupt, break or reinvent the totality of our genetic and life experience language programming is through conscious and/or corporal override – deprogramming, reprogramming or inserting new code and new languages into our mechanisms of perception, cognitive processing, context and systemic understanding.

The study of cognitive semiotics and linguistics, along with the study of social sciences and neurological/sensory sciences, provide the blueprint for the invention and ubiquitous adoption of a representational language capable of enabling participative knowledge synthesis and comprehensive systemic understanding.

I now ask you to work with us to organize a global project to hypothesize, prototype, test and ultimately teach a unique representational multimedia language that empowers everyone to better understand and apply complexity across the fabric of society. This effort would consist of applying Cognitive Semiotics and Linguistics to the invention of a visual and auditory computer aided language capable of taming and harnessing complexity. Such a language would ignite a participative revolution in human thought and action, improving lives, saving lives, empowering humans to transcend the severe limitations of words.

For a start, after 5 years of concentrated consideration, here are some of our imagined foundational principles that might contribute to the objective of inventing a process, toolset and language for systemic understanding based upon cognitive semiotics and linguistics:

1) The most effective mechanism for human beings to comprehend multiple dynamic variables and their exchanges and interactions across time and space involves the use of fixed ontological cellular representations filled with a simplified semiotic language (visual and auditory patterns that convey meaning either intuitively or through cognitive learning and synthesis) and its accompanying linguistics and creative expression. Drawing lines to connect variables is inherently ineffective. Having a simplified language composed of colors, shapes, ontological representations and fixed modelsets filled with pattern can be a way forward.

2) Any participative widespread adoption of a mechanism to address complexity requires trust. Trust is best accomplished through a transparent documentation of the objectives, processes, content, sources, research, science, verifications and validations comprising a cumulative mediated visual proof process showing the derivation of everything.

3) Current languages and forms of media must inextricably be linked to the evolution and ultimate foundation of a systemic semiotic language. Any new systemic communications system will not replace existing language structures, but rather serve as a high level representation of the meaning derived through the application of current languages to complexity, a labeling and tagging mechanism for all of the ontological variables, and a complementary language co-existing with current language and data visualization.

4) A new dynamic systemic semiotic language is not just an academic exercise even though it may evolve through academic institutions. A semiotic systemic multimedia language may be capable of surpassing all current forms of communication when it comes to:

 

1) complexity

2) extracting and communicating verifiable knowledge with alternate perspectives

3) synthesizing systemic understanding

4) understanding and operating any form of human system or enterprise

5) satisfying the objectives of the creator or user in the most effective manner

6) interacting with a computer screen

7) teaching and learning

 

 

 

This is a concept paper outline for your consideration.

Please contact:

 

Alan Yelsey

Educational Psychologist, USA

Skype alan.yelsey1

Phone 612.616.5430

Email alan at kvsstudio dot com

Website www.kvsstudio.com

 

wayfinderblue - Version 2

 

2 thoughts on “Douglas Rushkoff Introduces Our Big Bold Project – Dynamic Semiotics

  1. Regarding:
    The only way to intercept, redirect, disrupt, break or reinvent the totality of our genetic and life experience language programming is through conscious and/or corporal override – deprogramming, reprogramming or inserting new code and new languages into our mechanisms of perception, cognitive processing, context and systemic understanding.

    The study of cognitive semiotics and linguistics, along with the study of social sciences and neurological/sensory sciences, provide the blueprint for the invention and ubiquitous adoption of a representational language capable of enabling participative knowledge synthesis and comprehensive systemic understanding.

    I now ask you to work with us to organize a global project to hypothesize, prototype, test and ultimately teach a unique representational multimedia language that empowers everyone to better understand and apply complexity across the fabric of society. This effort would consist of applying Cognitive Semiotics and Linguistics to the invention of a visual and auditory computer aided language capable of taming and harnessing complexity. Such a language would ignite a participative revolution in human thought and action, improving lives, saving lives, empowering humans to transcend the severe limitations of words.

    The first paragraph copied here stands alone as a very good description of the 500-year process of colonization. Cultures (probably) most often do have direct shared (and / or individuated) spiritual experiences, and routes, methodologies and rituals for evoking them. Most indigenous cultures also have or had significant communication with animals and plants. These direct experiences are a very high expression of what it is to be truly human, and to express and realize the ‘higher self’. Abstraction does not suffice. The idea that technological man is a higher expression is hubris.

    The real value is likely to be gained from people who operate at this level creating their own semiotic extensions, agreeing on the best uses of these forms to support their purposes, and using them as the equivalent of a paper topographic map or a newsletter, as in, this is where I went and had communion, and this is the message we have to share. The ‘this is where I went’ coding may be different for every culture, and possibly every lineage.

    Someone pointed out that most First Nations clans have some knowledge, songs, dances, etc. that are forbidden to share with others. Access to that essentially requires becoming a full member of that group, which may or may not be possible. No one outside that group has a right to demand to be included in the understanding of that meaning, but those in the group have the right to use any and all tools to support their own process of maintaining and deriving value from it.

    Once the first paragraph is set aside / discarded, or comprehensively limited / corralled, the second and third paragraphs of the selection become very interesting.

    Following the lead of Professor Akinsolo Akiwowo, who developed and taught The Indigenization of Sociology at the University of Ife, Nigeria, social sciences can be stripped of their colonial aspects and outlooks, and used as tools to support the 5,000 to 7,000 cultures and languages which remain, out of about 10,000, at the beginning of agriculture, about 10,000 years ago, and 9,000 at the beginning of colonialism, 500 years ago. Professor Akiwowo focused on restoring Yoruba culture, which was his heritage. Some combination of griots, shamans, herbalists, people from the culture trained in the concept of indigenizing both sociology and anthropology, extremely skilled pastoralists, farmers, craftspeople, youth and young adults, and many others would work with their own cultural heritage, comparing their collective appreciation of it with other contributions to the process that the second and third paragraphs envision, and choosing one or more approaches that sustain the most meaning, in their terms. They could share the process and the result with others in their culture, and with cultures with which they have significant interactions, refine, re-test, refine, and eventually adopt.

    To the extent that they (especially the young, and ambassadors of various sorts) have interaction with industrial culture and society, having done the above work would give them a strong hand in using both words and semiotics to shape the way they choose to relate to the so-called ‘modern’ world around them — and to groups who would support them in maintaining possession and protection of their lands and practices, and in sustaining their culture.

    For those of us living in industrial cultures, encountering a profusion of reclaimed conceptual territory of thousands of cultures would be quite salutory (health-promoting). It would help establish boundaries on the presumptions of people who are disconnected with the lands and cultures of their own ancestors, prior to disruption by conquest or absorption, and have no idea when they first start exploring the world that other people have other, equally valid, worldviews, and that the fact that they have access to advanced weaponry and tactics does not give them any right to impose their will on those whom they encounter.

  2. The origin of all wealth and wisdom is from the earth and universe. Indigenous peoples, peoples rooted in the earth, learned patterns and systemics that guided their corporal and spiritual lives. The sensorial environment was their semiotic knowledge world. If a plant bore fruit yet needed water, and could be processed as food in a variety of ways that conserved energy and preserved it from decay, that was the visual systemic map. Color and texture and geo/concept maps guided their lives – for nurture and the pursuit of systemic understanding. Their language in many ways was their world and it worked until their model was drastically disrupted by others. We have literally become removed from the earth, distant from root understanding and from our communities at large, and the artificial abstract language we have inherited as our mechanism of thinking and organizing does not work holistically, or as a sustainable and effective mechanism to grapple with the evolution of artificial complexity and organic distance. The purity of embracing the earth and its living systems has become corrupted into layers of hierarchical distortion that leave institutions to fail at supplying us with meaning: health, love, community, sharing, caring, understanding, learning, teaching, experiencing, touching and creating. Yes, dynamic semiotics brings us back to our visual and auditory roots – creating a computer aided world that feels and works like an extension of earthly life, and extends the wisdom of indigenous peoples to be at one with all.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>