MidtermFall2023Exam

Which of the following is true of running a Spark application on Hadoop YARN? a) In Hadoop YARN mode, the RDDs and variables are always in the same memory space b) Running in Hadoop YARN has the advantage of having multiple users running the Spark interactive shell ¢) There are two deploy modes that can be used to launch Spark applications on YARN - client mode and cluster mode d) Irrespective of the mode, the driver is launched in the client process that submutted the job. An existing RDD, unhcrRDD contains refugee data from the UNHCR. It contains the following fields: (Country of residence, Country of origin, Year, Number of refugees). Sample data is shown below. Assume that number of refugees is of type Int and all other values are of type String. To get the count of all refugees by country of residence, use which of the following in Scala? Array(Array(Afghanistan, Pakistan, 2013, 34), Array(Albania, Algeria, 2013, 0), Array(Albania, China, 2013, 12).). a) val country = unhcrRDD.map(x=>(x(0),x(3))).reduceByKey((a,b)=>a+b) b) val country = unhcrRDD.map(x=>(x(0),1)).reduceByKey((a,b)=>a+b) ¢) val country = unhcrRDD.map(x=>x.parallelize()) Given the pair RDD country that contains tuples of the form ((Country, count)), which of the following is used to get the country with the lowest number of refugees in Scala? a) val low = country.sortByKey().first b) wval low = country.sortByKey(false).first ¢) val low - country.map(x=>(x. 2.x. 1)).sortByKey().first d) val low - country.map(x=>(x._2.x. 1)).sortByKey(false).first When building a standalone application, you need to create SparkContext. To do this in Scala, you would include which of the following within the main method? 2} wal confe= new SparkCoat().setigplane | “Asct ionsdpp”™ ) val so* nevw SpariContest jooaf ) B wval so= SparkContest().setigplane | “Asct Llonsipe™ ) val coafs SparkCoat(sc) Q wval so* sew SparkCostest () val conf= new SparCont | ).setipplane ( “Asct ionshpe™ )