Legodroid: A Type-Driven Library for Android and LEGO Mindstorms Interoperability †
- Legodroid unleashes the computational power of an external device hosting and running a program that communicates with the robot through its ABI. The application business logic entirely runs on the Android side. The interaction with the brick is seamless and it allows the programmer to focus on the program architecture and algorithms, rather than on communication and related mechanisms. The robot is programmed by means of one big callback—we call it the lego main. This is the only entry-point for interacting with sensors and motors connected to the brick. The benefits of this approach include:
- computing power: Android devices ranging from mobile phones to tablets are equipped with a much more powerful CPU than the EV3, enabling time-consuming algorithms to run on the mobile side;
- sensors: robot’s sensors can be managed by procedures written in a higher level language. Moreover, a number of devices such as cameras and microphones can be exploited by the programmer for adding additional eyes and ears to the LEGO robot, processing data coming from these sensors as well;
- development environment: Android Studio  is a powerful IDE with debuggers, code analyzers, and other tools aiding developers in writing apps;
- third party technologies: the entire Android SDK lies at the programmer’s fingertips, including its versatile UI/UX subsystem, reusable services performing with a number of common system-wide tasks, plus a great variety of third party libraries that are available for Android and are suitable for inter-operation with LEGO Mindstorms—e.g., OpenCV for Android  for bringing image recognition to the robot through the smartphone camera.
- Type-driven development relies on an accurate type design, code reuse, and polymorphism, requiring validation when compiling the code rather than writing algorithms with untyped or barely typed data. The basic idea is that “a strong type system can not only prevent errors, but also guide you and provide feedback in your design process” . Type-driven programming brings the type-safe coding discipline coming from the world of functional languages to the world of IoT programming and mainstream application development.
2. Related Work
- Flashing the brick ROM with a custom firmware is also an option, arguably addressed to those willing to take over the system and reprogram it from scratch.
- An application can connect to the EV3 brick through a TCP socket and start sending commands, i.e., structured streams of bytes, formatted according to the EV3 ABI specification defined in the EV3 Communication Developer Kit  On the EV3 side, a server process is constantly up and listening to incoming WiFi (At the time of writing, Legodroid does not support WiFi connections, as the Bluetooth counterpart is preferable in most cases. A WifiConnection class is expected by design though and will be added in a future update.) or Bluetooth connections, serving clients by processing incoming commands and sending replies, as documented in the EV3 Firmware Developer Kit 
2.1. leJOS and NXT
2.3. Other Solutions
2.4. LEGO Mindstorms as an Educational Environment for Learning Sensor Programming
3. Methodological Principles
- Use higher-order functions . Fine-grained custom behaviors can be formulated via lambda expressions, which shift the focus on parametric polymorphism rather than subtyping. The higher-order function approach has been adopted by mainstream languages in the recent years and is nowadays accepted by the OOP community as a right way for customizing the behavior of a generic function 
- Never allow the programmer declare uninitialized variables and force her to construct objects in a valid state. Nullness checking is crucial: adding Java annotations @NotNull and @Nullable, combined with an aggressive use of the final qualifier, raises the code quality in a sensible way . This has an impact on how classes and constructors are designed. Avoiding no-argument constructors discourages creating empty uninitialized objects that eventually have to be populated by calling setters. This in turn discourages unneeded mutable data , thus reducing the overall statefulness of a program, which is responsible for runtime errors due to state invalidity .
- Reduce side effects to the minimum. Mutable data structures in most imperative programs happen to be involuntary, since mutability is the default condition for variables and fields in most mainstream languages. Overuse of assignment is a common source of bugs, especially when concurrent code is involved, whereas immutable data tend to lift errors up to the type level . Manipulating immutable data types does not make code execution slower, since most modern languages rely on call-by-reference argument passing and data are never copied unless explicitly .
- Use strong types even for intermediate results. Languages with extensible records and variants  allow for an accurate representation of the results of temporary computations. In Java, an advanced use of types and generics  can literally guide the programmer to the correct implementation. Each computational step is represented by a strong type. Any invalid sequence of operations would be rejected by the compiler. Control flow becomes data flow; and data are ultimately validated by type checking .
4. Architecture of the Library
4.1. Package Structure
- Low level API. It deals with serialization and byte-level manipulation of commands for communicating with the EV3 brick according to the EV3 Communication Developer Kit specification. The comm sub-package, detailed in Section 4.3, contains the Bytecode class, aimed at building commands by appending op-codes and manipulating parameters at the byte level in a straightforward way. Users willing to extend the library with new commands can limit use of such low-level primitives to small self-contained methods.
- Mid level API. Class Api (For the sake of brevity, we may refer to the EV3.Api nested static class as Api) provides the core primitives for interacting with EV3, such as reading SI or PCT values from a sensor. The EV3 Firmware Development Kit defines these as half-baked data types translating, respectively, into float and short in Java. Extending the library at this level means to add new methods implementing EV3 instructions that are currently unsupported by Legodroid, manipulating arrays of floats or short according to the specification in Section 4 of the EV3 Firmware Developer Kit.
- High level API. The Api class offers a family of getter methods constructing strong-typed handles to sensors and motors defined in the plugs package. Such handles exhibit methods performing high-level operations over sensors and motors and are distinct classes within the plugs sub-package. Extending the library at this level means to extend the Api class with new methods constructing new handles, which provide the methods implementing new commands for the brick in the same way as classes in plugs do.
4.2. The Root Package: legodroid.lib
4.3. The Communication Package: legodroid.lib.comm
4.4. The Sensors and Motors Package: legodroid.lib.plugs
- ports are distinct enum types EV3.OutputPort and EV3.InputPort;
- minor flags representing motor polarity and type (Refer to Section 4.9 of the EV3 Firmware Development Kit, op-codes opOutput_Polarity and opOutput_Set_Type) are enum types as well;
- all sensor and motor classes inherit from a common superclass Plug<P>, where P represents the port type: this makes subclasses instantiate the generic P with some concrete type at inheritance time;
- sensors inherit from a common abstract class AbstractSensor: protected methods getPercent(), getPercent1(), getSi(), and getSi1() are commodities for quickly implementing actual sensor subclasses.
4.5. The Utilities Package: legodroid.lib.util
5. Type-Driven Patterns
- in order to access EV3 motors and sensors, the programmer needs an object of type Api, which can only be obtained as argument passed to the callback of type Consumer<Api> that must be provided to the EV3.run() method;
- an object of type EV3 is therefore needed, which represents a physical EV3 brick connected to the Android device and its constructor requires an object of type Channel;
- a Channel can be created only by means of an object of type Connection: e.g., class BluetoothConnection implements Connection<String, BluetoothChannel>, where the String type argument represents the peer and BluetoothChannel is the return type;
- a Connection requires the peer at construction time: e.g., BluetoothConnection requires a string with the EV3 brick name in order for Bluetooth pairing to take place.
- the set of operations can be translated into methods of a stateless object of type A;
- each state can be translated into an object of type for :
- each object holds the information for the state ;
- an object of type , for , can only be constructed by providing an argument of type , i.e., the previous state;
- the initial state , implemented by an object of type , must be constructed explicitly from scratch;
- objects of type A can only be constructed given an argument of type .
- synchronizing access to a resource by locking and unlocking a mutex;
- trapping exceptions by surrounding a function call with a try-catch block (Method Prelude.trap() in package legodroid.lib.util is an example of this usage of Pattern 2).
6. Advanced Features
6.1. Reusing EV3
- Once a lego main callback returns, another callback can be run—there can be up to one running lego main at any given time on the same EV3 object;
- the same EV3 object can be shared among different threads, allowing concurrent access to sensors and motors in a thread-safe way.
6.2. Safe Concurrency
6.3. Improving Sensor Accuracy
- Light This produces a color value either in RGB format or as an enumeration value supporting a small set of constant colors (At the time of writing, the RGB mode produces random readings on the EV3 even though it appears to be supported. Future versions of the firmware may fix this behavior and Legodroid would immediately start to return correct readings.).
- Gyroscope It either outputs the angle value in degrees or the rotational speed in degrees per second.
- Pressure This consists of a button that simply maps onto a boolean value.
- Ultrasound By emitting ultrasonic waves, this sensor measures the distance from solid obstacles either in centimeters or inches.
- at the sensor level, where each class representing a sensor (LightSensor in our case) can be customized and the relevant factory method overridden within the Api class;
- at the data-retrieval level, where each customized sensor subclass (AveragingLightSensor extends LightSensor in our case) can either add extra methods or override the behavior of the existing ones.
|Algorithm 1: Reconstructing inaccurate colors collected by the light sensor (Procedural Version).|
7.1. In-Depth Impact Evaluation
7.2. Usability Evaluation
- wheeled machines capable of processing the environment via sensors (both in the robot and in the Android device) in order to collect objects and avoid obstacles;
- an ink-jet printer capable of rendering an input image acquired by the smartphone camera, by moving a pen up and down on a scrolling paper;
- an mp3 player with a physical user interface: by moving levers and pressing buttons, the Android device plays/pauses a song, skips to the next track, raises the volume, etc.;
- a motorized crane capable of carrying and moving objects;
- a weight scale for small objects to explain the theoretical definition of weight as : makes use of the pressure sensor to detect when the equation is balanced while giving power to the motors lifting the object;
- a multiple safe deposit box, whose chambers can be accessed by means of a password on the companion mobile app, which activates motors for turning the safe interior and open the right door;
- a color Sudoku interactive solver;
- a pills dispatcher, splitting apart a bunch of pills on a color basis in different boxes, and delivering them according to a given schedule.
7.3. Limitations and Future Work
Conflicts of Interest
- Spanò, A.; Cortesi, A.; Zausa, G. Type-Driven Cross-Programming for Android and LEGO Mindstorms Interoperability. In Proceedings of the IFIP International Conference on Computer Information Systems and Industrial Management, Belgrade, Serbia, 19–21 September 2019. [Google Scholar]
- Legodroid. Available online: https://github.com/alvisespano/Legodroid/tree/sensors (accessed on 28 February 2020).
- Android Studio. Available online: https://developer.android.com/studio (accessed on 28 February 2020).
- OpenCV for Android. Available online: https://opencv.org/android (accessed on 28 February 2020).
- Type Driven Development. Available online: https://blog.ploeh.dk/2015/08/10/type-driven-development/ (accessed on 28 February 2020).
- ROBOLAB Reference Guide. Available online: http://www.legoengineering.com/robolab-programming-references (accessed on 28 February 2020).
- Resnick, M.; Maloney, J.; Monroy-Hernández, A.; Rusk, N.; Eastmond, E.; Brennan, K.; Millner, A.; Rosenbaum, E.; Silver, J.; Silverman, B.; et al. Scratch: Programming for all. Commun. ACM 2009, 52, 60–67. [Google Scholar] [CrossRef]
- leJOS Documentation. Available online: http://www.lejos.org (accessed on 28 February 2020).
- LEGO Mindstorms NXT Education Kit. Available online: https://www.generationrobots.com/media/Lego-Mindstorms-NXT-Education-Kit.pdf (accessed on 28 February 2020).
- EV3 Communication Developer Kit. Available online: https://le-www-live-s.legocdn.com/sc/media/files/ev3-developer-kit/lego%20mindstorms%20ev3%20communication%20developer%20kit-f691e7ad1e0c28a4cfb0835993d76ae3.pdf (accessed on 28 February 2020).
- EV3 Firmware Developer Kit. Available online: https://le-www-live-s.legocdn.com/sc/media/files/ev3-developer-kit/lego%20mindstorms%20ev3%20firmware%20developer%20kit-7be073548547d99f7df59ddfd57c0088.pdf (accessed on 28 February 2020).
- LEGO Mindstorms NXT Kit Download. Available online: https://www.lego.com/en-us/service/help/products/themes-sets/lego-education/lego-mindstorms-nxt-software-downloads-408100000007850 (accessed on 28 February 2020).
- leJOS PC API. Available online: https://lejos.sourceforge.io/nxt/pc/api/index.html (accessed on 28 February 2020).
- leJOS Documentation: Examples. Available online: http://www.lejos.org/nxt/nxj/tutorial/Android/Android.htm (accessed on 28 February 2020).
- Göbel, S.; Jubeh, R.; Raesch, S.L.; Zündorf, A. Using the Android platform to control robots. 2011. Available online: https://www.researchgate.net/profile/Albert_Zuendorf/publication/267985764_Using_the_Android_Platform_to_control_Robots/links/54b952e70cf253b50e2989cc/Using-the-Android-Platform-to-control-Robots.pdf (accessed on 28 February 2020).
- Manousaridis, K.; Mavridis, A.; Kalogiannis, G.; Anagnostopoulos, K. Introducing an innovative robot-based mobile platform for programming learning. In Proceedings of the 2015 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL), Thessaloniki, Greece, 19–20 November 2015. [Google Scholar]
- Nádvorník, J.; Smutnỳ, P. Remote control robot using Android mobile device. In Proceedings of the 2014 15th International Carpathian Control Conference (ICCC), Velke Karlovice, Czech Republic, 28–30 May 2014. [Google Scholar]
- Barnes, D.J. Teaching Introductory Java through LEGO MINDSTORMS Models. In Proceedings of the 33rd SIGCSE Technical Symposium on Computer Science Education, Cincinnati, KY, USA, 27 February–3 March 2002. [Google Scholar] [CrossRef]
- Klassner, F.; Anderson, S.D. Lego MindStorms: Not just for K-12 anymore. IEEE Rob. Autom Mag. 2003, 10, 12–18. [Google Scholar] [CrossRef]
- Williams, A.B. The qualitative impact of using LEGO MINDSTORMS robots to teach computer engineering. IEEE Trans. Educ. 2003, 46, 206. [Google Scholar] [CrossRef]
- Lund, H.H.; Pagliarini, L. Robocup jr. with lego mindstorms. In Proceedings of the 2000 ICRA, Millennium Conference, IEEE International Conference on Robotics and Automation, Symposia Proceedings (Cat. No. 00CH37065), San Francisco, CA, USA, 24–28 April 2000. [Google Scholar]
- Klassner, F. A Case Study of LEGO Mindstorms’TM Suitability for Artificial Intelligence and Robotics Courses at the College Level. In Proceedings of the 33rd SIGCSE Technical Symposium on Computer Science Education, Cincinnati, KY, USA, 27 February–3 March 2002. [Google Scholar] [CrossRef]
- Kim, S.H.; Jeon, J.W. Introduction for freshmen to embedded systems using LEGO Mindstorms. IEEE Trans. Educ. 2008, 52, 99–108. [Google Scholar] [CrossRef]
- McWhorter, W.I.; O’Connor, B.C. Do LEGO® Mindstorms® motivate students in CS1? In Proceedings of the 40th ACM technical symposium on Computer science education, Chattanooga, TN, USA, 4–7 March 2009.
- Designing with Types. Available online: https://fsharpforfunandprofit.com/series/designing-with-types.html (accessed on 28 February 2020).
- Damas, L.; Milner, R. Principal Type-Schemes for Functional Programs. POPL 1982, 82, 207–212. [Google Scholar]
- Types: Properties of Software Development. Available online: https://blog.ploeh.dk/2016/02/10/types-properties-software/ (accessed on 28 February 2020).
- Xi, H.; Pfenning, F. Dependent types in practical programming. In Proceedings of the 26th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, San Antonio, TX, USA, 20–22 January 1999. [Google Scholar]
- Brady, E. Programming and reasoning with algebraic effects and dependent types. In Proceedings of the 18th ACM SIGPLAN international conference on Functional programming, Boston, MA, USA, 25–27 September 2013. [Google Scholar]
- Gibbons, J. Design patterns as higher-order datatype-generic programs. In Proceedings of the 2006 ACM SIGPLAN workshop on Generic programming, Portland, OR, USA, 16 September 2006. [Google Scholar]
- Lambdas Are Not Functional Programming. Available online: https://medium.com/@johnmcclean/lambdas-are-not-functional-programming-63533ce2eb74 (accessed on 28 February 2020).
- Chalin, P.; James, P.R. Non-null references by default in Java: Alleviating the nullity annotation burden. In Proceedings of the European Conference on Object-Oriented Programming, Berlin, Germany, 30 July–3 August 2007. [Google Scholar]
- Chalin, P.; James, P.R.; Rioux, F. Reducing the use of nullable types through non-null by default and monotonic non-null. IET Softw. 2008, 2, 515–531. [Google Scholar] [CrossRef]
- Designing with Types: Making Illegal States Unrepresentable. Available online: https://fsharpforfunandprofit.com/posts/designing-with-types-making-illegal-states-unrepresentable (accessed on 28 February 2020).
- Haack, C.; Poll, E. Type-based object immutability with flexible initialization. In Proceedings of the European Conference on Object-Oriented Programming, Genova, Italy, 6–10 July 2009. [Google Scholar]
- Bloch, J. Effective Java; Pearson Education: New Delhi, India, 2016. [Google Scholar]
- Gaster, B.R.; Jones, M.P. A Polymorphic Type System for Extensible Records and Variants. 1996. Available online: http://www.cs.cmu.edu/aldrich/courses/819/papers/row-poly.pdf (accessed on 28 February 2020).
- Bracha, G.; Odersky, M.; Stoutamire, D.; Wadler, P. Making the future safe for the past: Adding genericity to the Java programming language. Acm Sigplan Not. 1998, 33, 183–200. [Google Scholar] [CrossRef]
- Zibin, Y.; Potanin, A.; Ali, M.; Artzi, S.; Kie|un, A.; Ernst, M.D. Object and reference immutability using Java generics. In Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering, Dubrovnik, Croatia, 3–7 September 2007. [Google Scholar]
- Ellis, B.; Stylos, J.; Myers, B. The factory pattern in API design: A usability evaluation. In Proceedings of the 29th International Conference on Software Engineering, Minneapolis, MN, USA, 20–26 May 2007. [Google Scholar]
- JDK 8 Documentation: Package java.util.function. Available online: https://docs.oracle.com/javase/8/docs/api/java/util/function/packagesummary.html (accessed on 28 February 2020).
- Allen, E.; Bannet, J.; Cartwright, R. A First-Class Approach to Genericity. In Proceedings of the 18th Annual ACM SIGPLAN Conference on Object-Oriented Programing, Systems, Languages, and Applications, Anaheim, CA, USA, 26–30 October 2003. [Google Scholar] [CrossRef][Green Version]
- JDK 8 Documentation: Type Inference. Available online: https://docs.oracle.com/javase/specs/jls/se8/html/jls-18.html (accessed on 28 February 2020).
- Welc, A.; Jagannathan, S.; Hosking, A. Safe Futures for Java. In Proceedings of the 20th Annual ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, San Diego, CA, USA, 16–20 October 2005. [Google Scholar] [CrossRef]
- Haller, P.; Prokopec, A.; Miller, H.; Klang, V.; Kuhn, R.; Jovanovic, V. Futures and Promises. 2012. Available online: http://docs.scala-lang.org/overviews/core/futures.html (accessed on 28 February 2020).
- Android FutureTask Class Documentation. Available online: https://developer.android.com/reference/java/util/concurrent/FutureTask (accessed on 28 February 2020).
- Android Executor Interface Documentation. Available online: https://developer.android.com/reference/java/util/concurrent/Executor (accessed on 28 February 2020).
- Mycroft, A. Polymorphic type schemes and recursive definitions. In Proceedings of the International Symposium on Programming, Toulouse, France, 17–19 April 1984. [Google Scholar]
- Subramaniam, V. Functional Programming in Java: Harnessing the Power of Java 8 Lambda Expressions. 2014. Available online: https://www.amazon.com/Java-Lambdas-Pragmatic-Functional-Programming-ebook/dp/B00J3B3J3C (accessed on 28 February 2020).
- Warburton, R. Java 8 Lambdas: Pragmatic Functional Programming; O’Reilly Media, Inc.: Newton, MA, USA, 2014. [Google Scholar]
- Nguyen, D.; Wong, S.B. Design patterns for lazy evaluation. In Proceedings of the 31st SIGCSE Technical Symposium on Computer Science Education, Austin, TX, USA, 7–12 March 2000. [Google Scholar]
- Ehringer, D. The dalvik virtual machine architecture. Tech. Rep. 2010, 4, 1–8. [Google Scholar]
- Zhang, L.; Krintz, C.; Soman, S. Efficient support of fine-grained futures in Java. In Proceedings of the International Conference on Parallel and Distributed Computing Systems (PDCS), San Francisco, CA, USA, 13 February 2006. [Google Scholar]
- Zhang, L.; Krintz, C.; Nagpurkar, P. Language and Virtual Machine Support for Efficient Fine-Grained Futures in Java. In Proceedings of the 16th International Conference on Parallel Architecture and Compilation Techniques, Brasov, Romania, 15–19 September 2007. [Google Scholar] [CrossRef][Green Version]
- Method References (Java Documentation). Available online: https://docs.oracle.com/javase/tutorial/java/javaOO/methodreferences.html (accessed on 28 February 2020).
- Ford, A.; Roberts, A. Colour Space Conversions; Westminster University: London, UK, 1998; pp. 1–31. [Google Scholar]
- Foley, J.D.; Van, F.D.; Van Dam, A.; Feiner, S.K.; Hughes, J.F.; Angel, E.; Hughes, J. Computer Graphics: Principles and Practice; Addison-Wesley Professional: Boston, MA, USA, 1996. [Google Scholar]
- Internal Commission on Illumination (CIE). Available online: http://cie.co.at (accessed on 28 February 2020).
- Granger, E.M. Is CIE L* a* b* good enough for desktop publishing? In Proceedings of the Device-Independent Color Imaging, International Society for Optics and Photonics, San Jose, CA, USA, 15 April 1994.
- Poynton, C.A. SMPTE Tutorial: “Gamma” and its Disguises: The Nonlinear Mappings of Intensity in Perception, CRTs, Film, and Video. SMPTE J. 1993, 102, 1099–1108. [Google Scholar] [CrossRef]
- Color Metrics. Available online: https://www.compuphase.com/cmetric.htm (accessed on 28 February 2020).
- Barak, M.; Zadok, Y. Robotics projects and learning concepts in science, technology and problem solving. Int. J. Technol. Des. Educ. 2009, 19, 289–307. [Google Scholar] [CrossRef]
- Bobtsov, A.A.; Pyrkin, A.A.; Kolyubin, S.A.; Shavetov, S.V.; Chepinskiy, S.A.; Kapitanyuk, Y.A.; Kapitonov, A.A.; Bardov, V.M.; Titov, A.V.; Surov, M.O. Using of lego mindstorms nxt technology for teaching of basics of adaptive control theory. IFAC Proc. Vol. 2011, 44, 9818–9823. [Google Scholar] [CrossRef][Green Version]
- Konvicka, J.; Kotyrba, M.; Volna, E.; Habiballa, H.; Bradac, V. Adaptive control of EV3 robot using mobile devices and fuzzy logic. In Proceedings of the International Conference on Information Science and Applications, Hong Kong, China, 25–27 June 2018. [Google Scholar] [CrossRef]
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Spanò, A.; Cortesi, A. Legodroid: A Type-Driven Library for Android and LEGO Mindstorms Interoperability. Sensors 2020, 20, 1926. https://doi.org/10.3390/s20071926
Spanò A, Cortesi A. Legodroid: A Type-Driven Library for Android and LEGO Mindstorms Interoperability. Sensors. 2020; 20(7):1926. https://doi.org/10.3390/s20071926Chicago/Turabian Style
Spanò, Alvise, and Agostino Cortesi. 2020. "Legodroid: A Type-Driven Library for Android and LEGO Mindstorms Interoperability" Sensors 20, no. 7: 1926. https://doi.org/10.3390/s20071926