经验首页 前端设计 程序设计 Java相关 移动开发 数据库/运维 软件/图像 大数据/云计算 其他经验
当前位置:技术经验 » Java相关 » Scala » 查看文章
Spark笔记(一):错误总结
来源:cnblogs  作者:等木鱼的猫  时间:2019/1/28 9:42:03  对本文有异议

1.转义字符:

  常见的replaceAll,split,mkstring中涉及到特殊字符的都要加上转义字符,比如str.split("\\|"),str.replaceAll("\\[", "")

     报错信息:

      java.util.regex.PatternSyntaxException: Unclosed character class near index 0

      java.util.regex.PatternSyntaxException: Unexpected internal error near index 1

 

2.kafka中数据还没来得及消费,数据就已经丢失或者过期了:

  [https://blog.csdn.net/yxgxy270187133/article/details/53666760]

  报错信息:

  org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets out of range with no configured reset policy for partitions:{...}

3.spark submit 内存参数太小 --executor-memory 8G \  --driver-memory 8G \

  报错信息:

   Application application_1547156777102_0243 failed 2 times due to AM Container for appattempt_1547156777102_0243_000002 exited with exitCode: -104
For more detailed output, check the application tracking page:https://host-10-31-4-246:26001/cluster/app/application_1547156777102_0243 Then click on links to logs of each attempt.
Diagnostics: Container [pid=5064,containerID=container_e62_1547156777102_0243_02_000001] is running beyond physical memory limits. Current usage: 4.6 GB of 4.5 GB physical memory used; 6.3 GB of 22.5 GB virtual memory used. Killing container.

 

原文链接:http://www.cnblogs.com/ShyPeanut/p/10318790.html

 友情链接: NPS  问卷模板