Mobile Devices And Communication Apps - Current trends and future directions

David: My name is David McNaughton and I’m here today with Sam Sennott, one of the developers of ProloQuo2Go. We are going to be talking with Howard Shane and Jessica Gosnell of the Boston’s Children’s Hospital. Hello. Howard & Jessica:Good Morning. David: We’re excited to have this opportunity to talk to you about some of the things that you’ve been seeing with mobile communication technologies in your work at Boston Children’s Hospital. It’s certainly been a period of incredible excitement and Howard, you’re involved in Apple’s Launch of the iPad 2. There’s been articles in the New York Times, so clearly this is a time when there’s a lot of excitement and anticipation about the use of these technologies. Howard, let’s start with you. Can you tell us a little bit about why you think there’s such excitement about this technology as a means of communication? Howard: Well, I think that the excitement is coming from the fact that it’s a mainstream technology. The price point is very attractive to people. There’s a great deal of applications known in the ‘biz’ as apps that are available and so it makes it more accessible to people – the screen size, the multiplicity of other applications, not just communication, but educational opportunities, the entertainment value providing rewards. It just has a whole multitude of opportunities and so I think all of those together makes it a very unique kind of technology. David: Now, it really has been extraordinary to see the reaction from families and teachers and the entire field. Howard: Yeah, it is. Sam: So, Jessica, you’re at Children’s Hospital of Boston, both of you. On a day-to-day basis, what kinds of things are you seeing related to people who use AAC and the iPad? Jessica: Yeah, I mean, I think both of us are seeing there’s probably not a session where the iPad isn’t really being talked about, at least brought up, or being used. Sam: Okay. Jessica: And everyone’s really excited about it and that’s been a big kind of clinical shift for us, probably, where people are either coming in with the equipment or… Sam:0:02:36.370,0:02:37.030 Sure. Jessica: …really have done research, heard about it, asked about it. And, so it kind of gets brought up immediately, as opposed to what you (referring to Howard) always say we have like the ‘recipe’ you know. People used to come here and we would show them devices and have things here and it’s kind of flipped where people are coming in with equipment. Sam: Okay. Jessica: That’s a big something we’re seeing daily. Howard: Let me just add. It’s not at all unusual for a child, especially we see it with children on the autism spectrum, for the family to come in with an iTouch and an iPad. Sam: Oh, interesting. Now, were you seeing this with traditional devices, where they would bring in, say, a Dynavox or another device? Howard: No, and I think… Jessica: Rarely Howard: Rarely Sam: 0:03:22.530,0:03:22.970 Okay Howard: And I think to Jessica’s point, people would come here because we had the secret sauce. Sam:Sure. Howard: But, now, families are coming in and they bring their iPad or their iTouch and they’re asking us, “Well what are the apps?” or “What do you think of this app or that app” so… Sam: Yeah. Howard: It’s changed the clinical model a bit. Sam: Yeah. Could you describe that, about your assessment process and your overall model? How does the iPad impact that? You’ve spoken a little bit about it, but how does it work? Howard: Well, I think Jess can speak to the strategy that we take and the work that she’s been doing specifically to keep us grounded, that it’s not so unusual that we have a whole different clinical approach. Sam: Oh, sure. If you’d like to take the same clinical approach, I think that Jessica really embodies that in her work, so I think she ought to talk about that. Sam: Great, great. Jessica: I think that the big focus has been everyone’s excited and they’re coming in with their iPads and their apps but really bringing it back to feature matching in good clinical assessment. So, it’s not unusual for me to say, “Oh, I’m really glad you went out and bought an iPad. I’m glad you have apps, but during today’s session, I’m probably going to use the iPad, but we’re going to look at other strategies as well.” So, really bringing it back to a feature matching approach and if the i-Device is the right platform then continuing that feature matching on to pick the appropriate app. Sam: Okay, so, it’s putting the iPad and various apps in its place as a tool that you can select from? Jessica: Right. Sam: Okay. Thanks. Howard: What’s also extraordinary is the cost factor too. Sam: Sure. Howard: So, when you make a suggestion for an application, perhaps it’s free, or it’s 3 or 4 dollars or at most it’s a couple of hundred dollars, that’s a very different conversation that you have with a family when you’re talking about a device that’s anywhere from 2,000 to 8,000 and then you get into the discussion about reimbursement and insurance and so forth. Not that I don’t think that third party payers need to get into this game, it just changes the communication. Sam: Sure, so I like what we’re hearing here that you’re utilizing this rich tradition of high quality assessment and feature matching, considering the iPad, but in some ways, it really changes that decision making paradigm where sometimes people might have access to these tools because of funding. They might be able to get them quicker or easier because of funding. Is that accurate? Howard: Yes. And as we’ve mentioned earlier, often it’s not a matter of funding because the families show up with device. That’s certainly quite different than historically what we’ve faced here in our center and across the world. Jessica: And I think in some ways it’s been nice getting…you know, how many people have come, they’ve waited six months for an assessment they get the assessment, you make the recommendations and then the first part’s filling out paperwork. Well that takes two weeks. Then they wait a month to get the device from the company. Then it’s a month-long trial. So not that we want to play down the trials or the importance of that. Sam: Oh, sure. Jessica: It’s just a faster process so on the way home I don’t know how many parents have stopped and just picked up the iPad on the way home. When they get home they sync it with their computer or they buy the apps and then they call you the next day, “Hey, we’re ready for therapy.” David: Right Jessica: And that’s exciting that people are kind of following, I feel like, following up with recommendations and really kind of getting on board really quickly. David: Can you walk us through a little bit more about that typical assessment activity. So, I don’t know if you could maybe, perhaps, think of a child that would perhaps be typical of, you know, some of the children that you see and walk us through what goes on within assessment activities? Howard: Well, I’ll start... Jessica: You probably do it differently, a little bit differently. Howard: It’s a little bit different in terms of our clinic work. I mostly see children on the autism spectrum and Jess sees children with more motor complications. So, for the child who has better motor control, usually the child with autism, in many ways the access issue isn’t as big a difficulty, or a problem, in the clinical process. So, they obviously need to learn some of the fine, refined movements of swiping and how to navigate, but that tends to be not a big problem for individuals on the autism spectrum. So, we don’t have the big motor issues that we would have for someone with a physical disability. As far as the determination of the communication application, we’re using a similar process. We need to know what level of representation. Can they read, can they spell, can they use text, does it need to be symbol based. If it’s symbol-based what level of symbolization? Do they use Black and white line drawings, could they use the Mayer-Johnson symbols, do they need to use photographs, because at this point in their life that’s what makes the most sense to them. So we have to use those kind of cognitive representational considerations. We have to consider their language abilities. We want the device to be able to allow them to build their language, and it’s not just an expressive device, it also hopefully is going to help them with their understanding of input. For the person on the autism spectrum, besides, or as an adjunct to, their communication is their understanding of the temporal events of the day, so we want to select applications that allow them to be able to put together their schedule, a visual schedule quite often. So, we want to consider all of that. We want to see the kinds of difficulties the child might have with understanding those temporal relationships, that we want to find applications that are suitable for them to understand that whole process. So, I guess the bottom line is that we need to really consider the child’s abilities and the difficulties that they have and then we try to match that with this whole tool set, which are the apps and the platform itself. And, if it’s a good match, then we make the recommendation. And then as Jess said, families will say, “Well, I’m stopping on the way home at the Apple store,” so that becomes a very different time frame than the old days. Here we go. David: Interesting. Thank you. Howard: Jess might want to say a bit about the motor issues and how- and that’s going to change but it’s still somewhat of a difficulty for the child with you know, considerable motor complications. Jess: Right. I mean I think that that is in some ways, I think one of the questions maybe down the line was going to be why would you not recommend an Ipad? And I think that the motor is a huge piece into why you’re not going to- how they access the device. And I think, you know, some of the current technology has better options for maybe increasing the dwell or now we have, everything like you said, is kind of developing. We’re getting key guards, there are more switch interfaces, but, its kind of all just still in the early works. Howard: But it’s no question that it’s coming. You know that the issues around switch placement is going to be similar then whether it’s an Ipad or any other communication device. However, that said, if I put my, take out my crystal ball I don’t think it will be long before switches might become things of the past because the camera recognition of the device is going to allow gesture recognition, so you know, if I move my arm like that that would be the switch access that previously required me physically touching some kind of a switch. Sam: Absolutely. Howard: We’re moving into a whole new generation. And it’s just incredibly exciting to think of the possibilities and they’re unfolding at a pretty rapid rate. David: How have you dealt with that information explosion? So clearly we’re in a very different world from the time when there was a small number of manufacturers and you would become familiar with your options and that whole question of feature matching was more simplified because there was fewer options to really consider. Now we’re in a world where you know, Sam, I think you were joking the other day that two communication apps with the same name came out in the same week so clearly people are generating new ideas at an incredible pace. How do you try to manage that information explosion? Or do you have any suggestions for our audience? Howard: In our model we feed Jess a lot of coffee…you know you hear about a lot of it but what we as a group, especially in our augmentative communication group, when new apps come out Jess kind of leads the charge and you know starts to discuss them and then we do an analysis and try to put it into a growing chart. You might want to tell them about that. Jess: I mean I think we’re all good in a sense of collaborating, you know people are googleing, hearing things from other people and just doing a good job of keeping each other in the loop so sending lots of emails- “hey this is an app. You should buy it, you should check it out, or its been loaded on this central iPad, you can go look at it, and then let’s discuss it on Wednesday and talk about what we think about it, what we like about it, what our, things we would want changed or who we could maybe use it for”. So I think it’s kind of a group effort with everybody just kind of talking a lot. But it’s, I mean I think, it’s also been really hard to keep up with, and I’m sure we’ve missed stuff, and I’m sure other people know things that we don’t just by the nature of how hard it is to keep up with. Howard: Now we’re also- we’re trying to create- Jess is growing a feature matching chart that tries to keep up with the apps as they come along. I think we’ve pretty much stayed on top of it. We’re out lecturing and talking about it, you know, in conferences and learning about it and reading about it, so it’s a process. There’s just more of it. And you’re right, it requires more of a time commitment, but I think eventually as it all settles into the new model of clinical care, I think we’ll find ways in which the apps as they become available will go through some review and the good features will be listed and we’ll put it back into a clinical model to match what the child needs with what the apps offer. Sam: Okay, great. So we want to shift to intervention. And we talked a good amount about assessment and this process of feature matching. Once these individuals and their families have a system, whether it be an iPad or another system, how are things working specifically with ongoing development of language, with ongoing supports for someone using an iPad with an app? Howard: Well, I think that in many ways there’s certainly some similarities. It’s not like we have a whole new paradigm for how we’re intervening. I think that the intervention, it seems to be used more often in various environments. There seems to be a greater willingness to do some of the programming. There are some other issues around intervention that I think are worth noting. First, because there’s so many exciting other opportunities on the iPad, it sometimes- at least for children who can become somewhat obsessed with the entertainment, they’re sometimes reluctant to go to their communication applications because they want to look at youtube, they want to play some of the interesting games and other puzzles, and you know, other applications that have more of an entertainment value. So that sometimes can be a problem. I think what’s also at least quite clear to me is that all of the intervention materials of the future are - anything that’s pictorial, that has a visual base, I think we’re going to see these materials going to an electronic format. To be fumbling through a card catalogue trying to find stimulus pictures for the intervention I think is going to change when you can organize all that in some photo library and bring up your materials. That’s certainly going to be somewhat different. Sam: I heard this great story a couple of weeks ago the Autism Cares foundation in Pennsylvania is hosting this seven weeks series of using your Ipad with your child for parents. And what I thought is so interesting is that they said that for the first time that they had seen it was almost all fathers coming in with their child. And I think that for all parents and caretakers there’s an interesting connection there for ongoing intervention and for like you said working with other apps. Have you seen much around that area of engagement or increased interest by families? Howard: Well I mean I think that because the technology is so mainstream people simply just feel more comfortable with it. Once they understand how to sync it up with their computer, how information or how the apps are downloaded, once they get comfortable with that piece, and that’s really, again that’s a very mainstream way of getting information and retrieving information, anyone that’s used iTunes understands the whole concept of, you know that’s taken over the world so that’s a big difference. Jess: And I think that using kind of fun motivating apps to get kids to use communication strategies, maybe their communication tool isn’t the iPad. Maybe it’s a communication notebook or just topic displays, but using the iPad platform to get parents to actually use those strategies has been a big shift, I would say, in something that I’m seeing a lot of where, oh I use you know, a pog app during the session with a topic display and the dad in the session downloads the app that I’m using and is like, “oh can you make a photocopy of that topic display? I’ll use it when I get home”. You know, little things like that that they’re just easy excitement, they’re used to the platform and really wanting to use it and carry it over at home. Howard: Yes another aspect of this that I find interesting is that there are things that we do, certainly with children with autism, if we just want to give them a visual cue, we suggest that families create folders in their iPhoto library that has to do with the playground or activities around the house, and you can retrieve that folder and show the photo in a very easy way. You grab that photo in an easy way and show it to the child. So it’s not necessarily an app, it’s just part of the feature set of the device. Sam: Yeah, now that most of the devices have cameras, the iPad 2, the iPod touch, and current iPhone, it’s really, in some ways the Tango had a camera but not many other devices had previously had cameras. Howard: Right, the camera, you know, for a child who is autistic, for example, and has their iPad 2, and they want to capture the daily events, they can snap a picture, come home with it, show their parents, and you know a picture truly says a thousand words. It has a thousand words, and so to be able to capture that, now we have something to talk about. So there’s all these interesting ways of using it that don’t even require apps it’s just part of the feature set of the device. David: Jessica, can you talk a little bit more about what this technology has meant for adding vocabulary and helping a child develop language over time? Has it gotten easier? Has it been about the same? What have you seen with this new technology in terms of the device being developed over time? Jessica: I think in some ways it parallels to what we were doing with dedicated devices. I feel like I'm not doing many things different. I'm still giving them a vocabulary checklist, and we're talking about what vocabulary . . . we still really want the device to be customized to meet the individual's needs. I do think that, in some ways, it's been a little harder where a lot of the apps currently have less of a language system. Maybe now that TouchChat is out there's more of like a word power system, but a lot of the apps out there aren't based off of maybe a lot of research or more language-based systems, so that has been kind of a struggle, talking about, OK, how are we going to add core words? What's the progression? Do we have to actually create this and then modify everything we've done and start from scratch, or is there a way to kind of really think big and hide buttons? So that has been kind of a struggle or something that really has to be discussed with parents in customizing and setting up the devices. Howard: I might want to add, also . . . it's like, again, putting on my futuristic goggles, and I don't have to look too far into the future. The ability, because the iPad has a GPS to be able to walk into one location versus another, and once the vocabulary is known to that particular setting, the device is going to be able to automatically switch the vocabulary depending on the setting in which you're located. And so that's almost a no-brainer in terms of what vocabulary access is going to be like in the future. And there's already an app that does some of that, but it's just simply going to get better. I mean, imagine what it's going to be like three, four, five years from now. Jessica: and I think people are continuing to think about the fact that we need some sort of language progression, and really looking at language in place, and so I think . . . the nice thing is you buy the app and you get the free updates, so as app developers are continuing to kind of collaborate with other people, I think there are just going to be more page sets that are really focusing on language intervention and how can we expand language. David: Well, let's talk a little bit then about representation and what we see as ways of representing the language that's within the system. Howard, I know you've been very involved in your research over time with the Visual Immersion Program. How has your research there informed what you're doing clinically? Howard: Well, we have a field study going on now where we're actually applying some of our visual principles in real settings and in schools, and we're trying to study the changes as a result of really an immersive environment in school, at home, and in the community. And what this technology allows us to do along with other electronic media, it just allows us to be able to access information more quickly, to be able to use different uses of the device in different settings. We do video modeling using an iPad, right in the environment. If we're using a static scene cue we can pull that out of a library. If we're using dynamic images, that's all contained on the device. The families again are much more comfortable in using the technology. It just makes it easier to use these emerging technologies, this electronic media, this mobile technology in different environments, and it just seems to be more accepted by more people. David: So where do you think this is all going next? And I'll turn this over to the three of you. I mean what would you most like to see, and are there any . . . Howard, you've discussed a few of the things you think are on the horizon. Where would you like to see this go and what are some of the things you see coming down the pike? Howard: There was a time when I could look ahead and really envision some opportunities that the future was going to hold. I mean I certainly see things that need to be created; I've mentioned movement detection for switch access and GPS, the opportunity to be able to locate where you are and to change vocabulary. Now that it has camera capability you can say, "Well I'd like it to get more rugged," or, "I'd like the speakers to be a little but louder," but I'm running out of ideas. It's just happening so quickly. It'll be exciting to see what the next generation is. I will say that the communication apps that we're going to see in sort of what I consider to be the next round is going to bring us into a whole new generation of technology. The technologies, the apps that exist now are not much unlike what's been developed for some of the dedicated devices, but the future holds some interesting . . . we're certainly going to see changes, and it's going to be exciting to watch that unfold. Sam: I have to chime in. Recently in the ASHA Perspectives special issue on mobile technologies, one of the writers, David Chapple, spoke about the need for alternate access. He's a man who uses a head-tracking system. I think that's such an important area to focus on. One of the great things is that Apple's developed a VoiceOver system for accessibility. Typically we think about that as benefiting individuals with vision impairments so that they can have it as a screen reader, but little did we know that it's a whole accessibility framework, so I'm really excited that VoiceOver is there, and as we harness it, it can be very easily harnessed for switch access, and then it can be very easily harnessed for all kinds of other alternate access, so that's one thing that I would say is really important going forward is not to have alternate access come later or as a secondary afterthought, but to have that be right up there alongside any developments we're making, because there are just too many people who have significant access needs that there are barriers created by having to touch a screen, especially a capacitive touchscreen. We've done some great things with stylus tools and adapting stylus tools but it's really important to get the alternate access down. We talked about switches, about head, eye gaze, various alternate pointing tools and selection tools will be available if we build it. Jessica: And I think too . . . the big thing also, like with restrictions, like a lot of people talk about locking kids into an app, and I would look at it more like just being able to turn off certain apps at a certain time, maybe with some sort of password or restricting them and saying, "OK, these are the five apps you get to use: your communication app, your Time Timer, your Visual Schedule . . . " so if you lock them in one app, you're kind of losing some of the functionality of the i-device platform, that integration of functions, so you know, just being able to restrict access to maybe the ones that they're perseverating on or the ones they're going into a lot, and maybe having that as, "Oh, you can work toward going into StarFall, but for right now I'm going to hide all of these apps." That, I'm sure, is hopefully something coming down the line. Sam: Yeah, a time lock would be great. School hours . . . there has to be an override button if you're going to go to recess and want to play some games with your friends, but other than that, some people need that as a structure. Jessica: Right. Sam: It's tempting to watch YouTube videos and play Angry Birds. Jessica: Right. Howard: Right, which is what Jess and I are going to do when we finish this conversation. David: Good timing. So as we come to a close, I guess I do just want to remind people about some of the resources that are out there. I'm going to list a few, and there may be additional ones that people want to mention. As Sam had mentioned, there was recently a, ASHA Perspectives special issue on the use of mobile technologies, and Jessica and Howard, I know you've contributed to that. There has recently been an RERC White Paper on the use of mobile technologies that is available from the AAC-RERC website, and there is also a full-length webcast by Howard, discussing his research on the Visual Immersion Program that's also available on the RERC website as well. Other resources that people wanted to mention? Jessica: The chart that everyone keeps talking about has updated. We were going under web reconstruction, so the new web page is now up there. They said that as soon as the new web reconstruction had launched, they will now work on putting up all of our materials. So, it should be up. They have all the materials, so it should be up shortly. Howard: That's going to be on the Children's Hospital Center for Communication Enhancement website. David: Great, so we'll link to that as well. Well, good. Sam: Thank you very much for taking the time. It's great to speak with you both. Howard: Nice talking to you. You take care.